Batches, a wrapped host collections of primitive data, providing fast data processing.

Batch is an abstraction for host specific low level data containers that are used by host in bulk data processing and manipulation, such as I/O or networking.

Batch support for given item type is host specific. For better interop, some batch methods should be invoked through host batch manager. This avoids reflection and simplify user code. When you know the concrete batch type, it is best to directly call host methods of this type.

Batches can be created from existing arrays or they can be allocated. They support cloning and reducing.

When reducing batches, the reduction function may not store received batch between two calls, as batches are usually reused (cleared and filled with next data).

batch

Available since version 1.0 (view source)

not referred automatically

RETURNS TRANSDUCER

Usage:
  • (batch requested-type size-hint)

  • (batch requested-type size-hint coll)

Type signatures:
  • ((U nil Class+ Type clojure.lang.Keyword) ⨯ (Maybe Integer+)) → Transducer

  • ((U nil Class+ Type clojure.lang.Keyword) ⨯ (Maybe Integer+) ⨯ []) → IRed

Returns a transducer that batches given step items into batches of requested-type, which may be a type, class or a keyword, and of size-hint size. Returns collection recipe if coll is given.

batch-manager

Available since version 1.0 (view source)

Usage:
  • (batch-manager item-type)

Type signature:
  • ((U nil Class+ Type)) → (Maybe BatchManager)

Returns the batch manager for a given item-type, or nil, if host does not support batches of item-type.

batch-manager-from

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (batch-manager-from batch)

Type signature:
  • (AnyBatch) → BatchManager

Returns batch manager from an existing batch.

batch-on

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (batch-on arr begin end)

  • (batch-on bm arr begin end)

Type signatures:
  • (AnyArray ⨯ Int ⨯ Int) → AnyBatch

  • (BatchManager ⨯ AnyArray ⨯ Int ⨯ Int) → AnyBatch

Returns batch based on given host array arr and begin and end range in that array. Uses bm if given, or creates one from array’s item type. Throws if host does not support batches on a given array.

See also: batch-support?

batch-support?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (batch-support? item-type)

Type signature:
  • ((U nil Class+ Type)) → Boolean+

Returns true if host supports batches of item-type, otherwise returns false.

decide-item-type

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (decide-item-type requested-type available-type)

Type signature:
  • ((U nil Class+ Type) ⨯ (U nil Class+ Type)) → (U nil Class+ Type)

Returns concrete item type, or nil if any type will suffice.

default-batch-size

Available since version 1.0 (view source)

not referred automatically

VAR of type clojure.lang.Var

A dynamic var holding default batch size.

item-types-match?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (item-types-match? requested-type available-type)

Type signature:
  • ((U nil Class+ Type) ⨯ (U nil Class+ Type)) → Boolean+

Returns true if item types match, false otherwise.

malleable

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (malleable coll)

Type signature:
  • ([]) → IBatchedRed

Returns batchable collection which takes size-hint more seriously when reduced.

Useful for cases where the cost of processing many batches is greater than the cost of merging batched to bigger ones. (one such case is writing to file).

provide-batch-manager

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (provide-batch-manager bm batch)

Type signature:
  • ((Maybe BatchManager) ⨯ AnyBatch) → BatchManager

If batch manager bm is nil, finds and returns batch manager from a given batch. Returns bm otherwise.

provide-batch-size

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (provide-batch-size size-hint)

Type signature:
  • ((Maybe Integer+)) → Int

Returns batch size based on given optional size-hint integer. Returned batch size is at least big as size-hint.

This promise may be dropped in later versions.

select-item-type

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (select-item-type item-type)

  • (select-item-type requested-type available-type)

Type signatures:
  • ((U nil Class+ Type)) → (U nil Class+ Type)

  • ((U nil Class+ Type) ⨯ (U nil Class+ Type)) → (U nil Class+ Type)

Returns concrete item type from a given requested and available types. Returns default item type (byte) if any type would suffice.