Fast parser engine with helper functions for tokenizer and parser machines.

Primary

LeftoverPair

Available since version 1.0 (view source)

not referred automatically

TYPE

Extends: IRed

Type for passing leftovers from a tokenizer.

See also: leftover?

container-parser

Available since version 1.0 (view source)

not referred automatically

MACRO (container-parser name close-token separator-token empty-container)

Constructs parser machine for container data types, with corresponding constructor. empty-container must be editable.

eof-handler

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (eof-handler machine config val m)

Type signature:
  • (Any ⨯ {} ⨯ Any ⨯ String+) → Any

Returns machine if (:incomplete-mode config) is set to :ignore, returns val if (:incomplete-mode config) is set to :keep, otherwise throws with message m. This helper function is used inside parser machine for default handling of -parse-eof! and -analyze-eof!.

See also: perror

ignore-token

Available since version 1.0 (view source)

not referred automatically

VAR of type IgnoreParser

A parser machine which ignores itself.

leftover

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (leftover token batch)

Type signature:
  • (Any ⨯ AnyBatch) → LeftoverPair

Returns composite result of token and batch used as a return value in tokenizers.

See also: leftover?

leftover?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (leftover? x)

Type signature: Predicate

Returns true if object x is an instance of LeftoverPair type, false otherwise.

See also: LeftoverPair

literal-tokenizer

Available since version 1.0 (view source)

not referred automatically

MACRO (literal-tokenizer name contents value)

Constructs tokenizer machine for data literals, with corresponding constructor.

See also: skipping-tokenizer

parser-engine

Available since version 1.0 (view source)

not referred automatically

RETURNS TRANSDUCER

Usage:
  • (parser-engine machine-factory)

  • (parser-engine machine-factory coll)

Type signatures:
  • (IParserMachineFactory) → Transducer

  • (IParserMachineFactory ⨯ []) → IRed

A transducer which parser step items into a parsed values, using machine-factory to control parsing.

parser-machine?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (parser-machine? x)

Type signature: Predicate

Returns true if object x directly implements IParserMachine protocol, false otherwise.

perror

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (perror & ms)

Type signature:
  • ((Va String+)) → nil

Throws parser engine error with a given messages.

See also: eof-handler

skipping-tokenizer

Available since version 1.0 (view source)

not referred automatically

MACRO (skipping-tokenizer name n value)

Constructs skipping tokenizer machine, with corresponding constructor. Skips n items from the input and then returns value.

See also: literal-tokenizer

string-literal-constructor

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (string-literal-constructor invalid-pred escape-fn octal-support?)

Type signature:
  • (Predicate ⨯ AnyFn ⨯ Boolean+) → AnyFn

Returns tokenizer machine constructor for string literal tokenizer machine supporting unicode and octal escapes, using escape-fn to translate escapes, invalid-pred to check for invalid input and octal-support? to toggle support for octal escape sequences.

See also: literal-tokenizer

tokenizer-engine

Available since version 1.0 (view source)

not referred automatically

RETURNS TRANSDUCER

Usage:
  • (tokenizer-engine machine-factory)

  • (tokenizer-engine machine-factory coll)

Type signatures:
  • (IParserMachineFactory) → Transducer

  • (IParserMachineFactory ⨯ []) → IRed

A transducer which process step values into tokens, using machine-factory to control tokenizing.

See also: parser-engine

tokenizer-machine?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (tokenizer-machine? x)

Type signature: Predicate

Returns true if object x directly implements ITokenizerMachine protocol, false otherwise.

See also: ITokenizerMachine

Lazy

drop-one

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (drop-one parents level-limit coll)

Type signature:
  • (IPersistentList ⨯ Integer+ ⨯ IRed) → ISeq

Returns a seq of tokens with first processed value removed. Note that this fn automatically skips tokens handled by parser machines (e.g. containers)!

drop-until-token

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (drop-until-token parents pred item-limit level-limit coll)

Type signature:
  • (IPersistentList ⨯ AnyFn ⨯ Integer+ ⨯ Integer+ ⨯ IRed) → ISeq

Returns a lazy seq of tokens with all tokens before pred returns true removed. Note that this fn automatically skips tokens handled by parser machines (e.g. containers)!

lazy-item-limit

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (lazy-item-limit config)

Type signature:
  • ({}) → Int

Returns container item limit based on given config map.

See also: lazy-level-limit

lazy-level-limit

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (lazy-level-limit config)

Type signature:
  • ({}) → Int

Returns container level limit based on given config map.

lazy-parser

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (lazy-parser parents level-limit coll)

Type signature:
  • (IPersistentList ⨯ Integer+ ⨯ IRed) → ISeq

Returns a lazy seq of parsed values from coll, within the context of parents. Throws if level-limit has been reached.

lazy-parser-engine

Available since version 1.0 (view source)

not referred automatically

RETURNS TRANSDUCER

Usage:
  • (lazy-parser-engine machine-factory coll)

Type signatures:
  • (ILazyParserMachineFactory) → Transducer

  • (ILazyParserMachineFactory ⨯ []) → IRed

Returns a lazy seq of parsed input collection coll, with parser engine using machine-factory to control parsing.

See also: parser-engine

lazy-parser-machine?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (lazy-parser-machine? x)

Type signature: Predicate

Returns true if object x directly implements ILazyParserMachine protocol, false otherwise.

max-lazy-level-limit

Available since version 1.0 (view source)

not referred automatically

VAR of type Var

A dynamic var holding maximum level limit in lazy parsers. Used to prevent blowing the stack.

See also: lazy-level-limit

process-one

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (process-one parser-machine config coll parents)

Type signature:
  • (IParserMachine ⨯ {} ⨯ IRed ⨯ IPersistentList) → [Any ISeq]

Returns a pair of parsed value and rest of seq.

See also: drop-one, take-one

take-one

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (take-one parents level-limit coll)

Type signature:
  • (IPersistentList ⨯ Integer+ ⨯ IRed) → Any

Returns first parsed value from coll. Throws if level-limit has been reached.

take-until-token

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (take-until-token parents pred item-limit level-limit coll)

Type signature:
  • (IPersistentList ⨯ AnyFn ⨯ Integer+ ⨯ Integer+ ⨯ IRed) → ISeq

Returns a lazy seq of parsed values from coll, up until pred returns true. Throws if either level-limit or item-limit has been reached.