trait PartitionGroupSinker extends AnyRef
Base sinker responsible for loading a fixed set of Kafka topic partitions to persistent storage. Implementers must define committed stream position lookup from the underlying storage upon initialization.
- Alphabetic
- By Inheritance
- PartitionGroupSinker
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def close(): Unit
Cleans up and closes the sinker.
- abstract val groupName: String
Name of the partition group, can be used in metrics and logs to differentiate between group sinkers.
- abstract val groupPartitions: Set[TopicPartition]
The Kafka topic partitions that this sinker is responsible for.
- abstract def heartbeat(): Unit
Notifies the sinker that record consumption is still active.
Notifies the sinker that record consumption is still active. Gives the sinker an opportunity to perform flushing with very low traffic streams.
- abstract def initialize(kafkaContext: KafkaContext): Map[TopicPartition, Option[StreamPosition]]
Initializes the sinker and returns the stream positions where topic partitions should be reset to before loading.
Initializes the sinker and returns the stream positions where topic partitions should be reset to before loading. Should be called once before any subsequent calls to write. This is most likely a blocking call since it queries Kafka/storage for offsets.
- kafkaContext
Kafka context to use when looking up / committing offsets to Kafka, if needed.
- returns
The initial loader positions in the owned topic partitions. If no position is returned the position will not be reset explicitly, meaning that the consumer will either reset it to the earliest/latest based on the configuration value of
auto.offset.reset, or will reset to the offset stored in Kafka, if any.
- abstract def write(record: StreamRecord): Unit
Writes a given stream record to storage.
Writes a given stream record to storage.
Calling this method does not ensure that the record will be flushed to storage, e.g. the sinker might implement batching thus delaying the actual storage.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()