Type Members

1.  trait RecordBatch extends AnyRef

   A base trait representing a batch of records.

2.  abstract class RecordBatchBuilder[+B <: RecordBatch] extends AnyRef

   A record batch builder, the base implementation takes care of keeping track of contained record ranges.

   A record batch builder, the base implementation takes care of keeping track of contained record ranges. Concrete implementations should additionally implement actual batch construction.

   B

   Type of the batches being built.

3.  trait RecordBatcher[+B <: RecordBatch] extends AnyRef

   A record batcher that provides new record batch builders.

   A record batcher that provides new record batch builders.

   B

   Type of record batches being built.

4.  class RecordBatchingSink[+B <: RecordBatch] extends PartitionGroupingSink

   A Sink that uses a specified RecordBatcher to construct record batches and commit them to a specified RecordBatchStorage once they are ready.

   A Sink that uses a specified RecordBatcher to construct record batches and commit them to a specified RecordBatchStorage once they are ready. It is a partition grouping sink, meaning records from each partition group are added to different batches and processed separately. Batch commits to storage are queued up and performed asynchronously in the background for each partition group. The commit queue size is configurable and commits block if the queues get full.

   B

   Type of record batches.

5.  class RecordBatchingSinker[B <: RecordBatch] extends PartitionGroupSinker with Logging with Metrics

   A PartitionGroupSinker that accumulates records to batches and stores them to some storage once ready.

   A PartitionGroupSinker that accumulates records to batches and stores them to some storage once ready.

   B

   Type of record batches persisted to storage.

6.  trait RecordFormatter[+R] extends AnyRef

   A formatter for mapping source records to R typed records.

   A formatter for mapping source records to R typed records.

   R

   Type of records being formatted to.

7.  trait RecordPartitioner[-R, +P] extends AnyRef

   Base trait for defining a record partitioning strategy, e.g.

   Base trait for defining a record partitioning strategy, e.g. by day or by country, etc.

   R

   Type of formatted records, i.e. the one being written to storage.

   P

   Type of the partition.

8.  trait RecordStreamWriter[-R] extends AnyRef

   An abstract writer for a stream of records of type R, most probably backed by some storage, e.g.

   An abstract writer for a stream of records of type R, most probably backed by some storage, e.g. a file.