Computer and network resources, systems, scopes.

dunaj.resource API is a very early experiment with incomplete functionality. Expect rough edges.

Basic characteristics of a resource is its limited availability within a computer system. Additional informations are available in the resource specification.

User acquires a resource with acquire! function. Resource is a stateful object and is released within a given scope. Resource can be in one of following states:

  • OPEN (dunaj.state/open?)

  • FAILED (dunaj.error/error)

  • CANCELLED (dunaj.state/cancelled?)

  • CLOSED (not observable)

  • RELEASING (not observable)

State transitions:

  • after acquiring: OPEN or FAILED

  • when working with resource: OPEN → OPEN, OPEN → FAILED

  • evaluation hits end of scope: OPEN → RELEASING

  • after release is done: RELEASING → CLOSED, RELEASING → FAILED

  • cancelling the release process: RELEASING → CANCELLED.

Resources with composite bands implement classic collection protocols (IIndexed, ILookup, ICounted). Resources where individuals items can be sent/received implement ISourcePort and ITargetPort. If resource contains single value, it can implement IReference or IBlockingReference. Resources can implement custom error handling.

Primary

acquirable?

Available since version 1.0 (view source)

Usage:
  • (acquirable? x)

Type signature: Predicate

Returns true if object x satisfies IAcquirableFactory protocol, false otherwise.

acquire!

Available since version 1.0 (view source)

Usage:
  • (acquire! factory)

  • (acquire! factory uri-or-map & options)

Type signatures:
  • (IAcquirableFactory) → Any

  • (IAcquirableFactory ⨯ Any ⨯ (Va Any)) → Any

Acquires object from a given factory, pushes it into current scope if it is releasable and returns it.

Uses :scope entry in factory to customize scope pushing process. Classic scope keys are :timeout, :timeout-mode and :mode.

Special :scope option called :release-fn overrides releasing. If set, it must contain two args fn accepting weak reference to acquired object and scope options. This custom function will then be called when releasing instead of standard release process, and it can return nil or other releasable object (or reference to one), which will then be released.

Throws if object cannot be acquired.

1
2
3
4
5
6
(ns foo.baz
  (:api dunaj)
  (:require [dunaj.resource :refer [size]]))

(with-scope
  (size (acquire! (file "src/clj/dunaj/resource.clj"))))

exchange!

Available since version 1.0 (view source)

Usage:
  • (exchange! resource coll)

Type signature:
  • (IWritable ⨯ IRed) → IRed

Writes coll into resource. Without waiting for write! to finish, returns result of read!-ing the resource.

While the write! exception is eaten, the write error results in the resource going into failed state with exception retrievable with error fn and returned reducible throwing when reduced.

Should not be used for non-blocking resource, as the write could postpone without a possibility to advance.

format

Available since version 1.0 (view source)

Usage:
  • (format resource formatter-factory)

  • (format resource parser-factory printer-factory)

Type signatures:
  • ((I IReadable IWritable) ⨯ (I IParserFactory IPrinterFactory)) → (I IReadable IWritable)

  • ((I IReadable IWritable) ⨯ IParserFactory ⨯ IPrinterFactory) → (I IReadable IWritable)

Returns a readable and writable resource formatted with a given formatter factory for both reading and writing. May also supply parser and printer factories separately.

1
2
3
4
5
6
7
8
9
10
(with-io-scope
  (str (take 50 (-> (resource "http://example.com")
                    acquire!
                    (format utf-8)
                    read!))))
;;=> "<!doctype html>\n<html>\n<head>\n    <title>Example D"

;; using slurp convenience function
(with-scope (str (take 50 (slurp "http://example.com"))))
;;=> "<!doctype html>\n<html>\n<head>\n    <title>Example D"

read

Available since version 1.0 (view source)

Usage:
  • (read x)

Type signature:
  • ((U String+ Uri IImmutableReadable)) → IRed

Returns finite collection recipe attached to the object x.

1
2
3
4
5
6
7
8
9
10
(def v
  (with-io-scope
    (vec (read (resource "http://md5.jsontest.com/?text=Dunaj")))))
;;=> #'foo.baz/v

v
;;=> [123 10 32 32 32 34 109 100 53 34 58 32 34 98 52 99 53 98 54 53 97 50 55 57 98 100 49 97 51 57 49 51 55 49 50 100 57 55 97 55 52 55 50 53 54 34 44 10 32 32 32 34 111 114 105 103 105 110 97 108 34 58 32 34 68 117 110 97 106 34 10 125 10]

(parse-whole json (parse utf-8 v))
;;=> {"md5" "b4c5b65a279bd1a3913712d97a747256", "original" "Dunaj"}

See also: read!, readable?, slurp

read!

Available since version 1.0 (view source)

Usage:
  • (read! x)

Type signature:
  • (IReadable) → IRed

Returns collection recipe attached to the object x. Returned collection recipe must be reduced inside io! block. The returned collection recipe may be infinite and may block.

If the resource is seekable, the position after reducing collection recipe from read! is undefined.

Reducing collection while it is being released may end by reducing ending or by I/O exception. It is thus advised to either expect an exception or to stop reducing before releasing the resource. May return postponed object if x is in a non-blocking mode.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
(ns foo.baz
  (:api dunaj)
  (:require [dunaj.resource.loopback :refer [loopback]]))

(def p (grab-scope (acquire! (loopback))))
;;=> #'foo.baz/p

(def res (key p))
;;=> #'foo.baz/res

(def scope (val p))
;;=> #'foo.baz/scope

(thread (io! (dored [x (lines (parse utf-8 (read! res)))]
                    (println! x)))
        (println! "done"))
;;=> #<ManyToManyChannel [email protected]>

(write! res (print utf-8 "lorem ipsum dolor\nsit amet\n"))
;;=> 27

(write! res (print utf-8 "consectetur adipiscing elit\n"))
;; lorem ipsum dolor
;; sit amet
;;=> 28

(release-scope! scope)
;; consectetur adipiscing elit
;; done
;;=> {:items ({:value #<WeakReference [email protected]>})}

read-one!

Available since version 1.0 (view source)

Usage:
  • (read-one! x)

Type signature:
  • (IReadable) → Any

Returns first read item from x, potentionally discarding other items. If the x is seekable, the position after read-one! is undefined. May return postponed object if x is in non-blocking mode.

See also: read!, readable?

readable?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (readable? x)

Type signature: Predicate

Returns true if object x directly implements IReadable protocol, false otherwise.

See also: IReadable, read!, read

resource

Available since version 1.0 (view source)

Usage:
  • (resource x & {:as opts})

Type signature:
  • (Any ⨯ Any) → (Maybe IAcquirableFactory)

Returns an acquirable factory based on given x, passing opts into it. Uses uri scheme to map resource type from currently registered resource providers. Returns nil if resource type cannot be found.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
(ns foo.baz
  (:api dunaj)
  (:require [dunaj.resource.loopback :refer [loopback]]))

;; matches uri scheme to the list of registered resource factories
(resource "http://example.com")
;;=> #dunaj.resource.http.HttpResourceFactory{:uri "http://example.com", :proxy nil, :secure? false, :allow-ui? nil, :timeout nil, :use-caches? nil, :follow? nil, :request-properties nil, :request-method nil, :chunked-streaming nil, :hostname-verifier nil, :ssl-context nil, :batch-size nil}

;; no scheme means file resource
(resource "file.txt")
;;=> #dunaj.resource.file.FileResourceFactory{:uri "file.txt", :mode [:read :write :create], :batch-size nil, :file-system nil, :working-directory nil}

;; another example, a client tcp connection
(resource "tcp://localhost:8081")
;;=> #dunaj.resource.tcp.TcpResourceFactory{:uri "tcp://localhost:8081", :remote-address nil, :remote-port nil, :local-address nil, :local-port nil, :batch-size nil, :non-blocking? nil, :keep-alive? false, :in-buffer-size nil, :out-buffer-size nil, :linger nil, :no-delay? false, :reuse? nil, :selector-provider nil}

slurp

Available since version 1.0 (view source)

Usage:
  • (slurp res)

  • (slurp res parser)

Type signatures:
  • (IImmutableReadable) → IRed

  • (IImmutableReadable ⨯ IParserFactory) → IRed

Returns a utf-8 collection recipe of data read from resource factory res, which was opened for immutable reading. May supply custom parser if encoding other than utf-8 is needed.

1
2
(with-scope (str (take 50 (slurp "http://example.com"))))
;;=> "<!doctype html>\n<html>\n<head>\n    <title>Example D"

spit!

Available since version 1.0 (view source)

Usage:
  • (spit! res coll)

  • (spit! res coll mode)

Type signatures:
  • (Any ⨯ IRed) → Integer+

  • (Any ⨯ IRed ⨯ Any) → Integer+

Writes coll to the res, returning number of bytes written. Uses utf-8 printer. Default mode is :append.

1
2
3
4
5
6
7
;; append by default
(with-scope (spit! "out.txt" "lorem ipsum"))
;;=> 11

;; custom mode
(with-scope (spit! "out.txt" "lorem ipsum" :truncate))
;;=> 11

spit-with!

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (spit-with! res printer coll)

  • (spit-with! res printer coll mode)

Type signatures:
  • (Any ⨯ IPrinterFactory ⨯ IRed) → Integer+

  • (Any ⨯ IPrinterFactory ⨯ IRed ⨯ Any) → Integer+

Writes coll to the res using printer for printing, returning number of bytes written. Default mode is :append.

transform

Available since version 1.0 (view source)

Usage:
  • (transform resource xform)

  • (transform resource read-xform write-xform)

Type signatures:
  • ((I IReadable IWritable) ⨯ Any) → (I IReadable IWritable)

  • ((I IReadable IWritable) ⨯ Any ⨯ Any) → (I IReadable IWritable)

Returns a readable and writable resource transformed with given transducer xform for both reading and writing. May also supply reading and writing transducer separately.

See also: format, exchange, read!, write!

writable?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (writable? x)

Type signature: Predicate

Returns true if object x directly implements IWritable protocol, false otherwise.

write!

Available since version 1.0 (view source)

Usage:
  • (write! x coll)

  • (write! x xform coll)

Type signatures:
  • (Any ⨯ IRed) → (U Integer+ Postponed)

  • (Any ⨯ Any ⨯ IRed) → (U Integer+ Postponed)

Writes coll into object x and returns number of items written. Takes an optional transducer xf.

Blocks or returns postponed object if in non-blocking mode. Throws if I/O error occurs. Writing while the resource is being released ends by I/O exception too. It is thus advised to expect an exception or to finish writing before releasing the resource.

write-one!

Available since version 1.0 (view source)

Usage:
  • (write-one! x val)

Type signature:
  • (Any ⨯ Any) → (U Integer+ Postponed)

Writes val into object x and returns number of items written. Takes an optional transducer xf. Blocks or returns postponed object if in non-blocking mode.

Throws if I/O error occurs. Writing while the resource is being released ends by I/O exception too. It is thus advised to expect an exception or to finish writing before releasing the resource.

See also: writable?, write!

Scope

grab-scope

Available since version 1.0 (view source)

MACRO (grab-scope & body)

Executes body in a new scope. Returns pair [ret scope-atom]. Takes an optional existing scope map or scope atom or defaults map as first arg. Following keys are valid:

  • :pervasive - Boolean, defaults to nil

  • :timeout - nil (blocks, default)/IDuration

  • :timeout-mode - nil (ignore, default)/:cancel/:throw

Note if pervasive is enabled, the returned atom can be updated by child threads even after grab-scope finishes. Mutates first arg if it is atom and there are more than one args. Throws when body throws, releasing scope with :fail mode.

in-scope?

Available since version 1.0 (view source)

Usage:
  • (in-scope?)

Type signature:
  • () → Boolean+

Returns true if there is a current scope present, false otherwise.

See also: with-scope

releasable?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (releasable? x)

Type signature: Predicate

Returns true if object x satisfies IReleasable protocol, false otherwise.

release-scope!

Available since version 1.0 (view source)

Usage:
  • (release-scope! scope)

  • (release-scope! scope mode)

  • (release-scope! scope mode ex)

Type signatures:
  • ((U {} IAtomic)) → {}

  • ((U {} IAtomic) ⨯ Keyword) → {}

  • ((U {} IAtomic) ⨯ Keyword ⨯ (Maybe java.lang.Throwable)) → {}

Releases scope based on mode and returns scope map. scope can either be a normal scope (map), or an atom holding scope map. mode is one of :exit, :success, :fail and defaults to :exit.

Throws if releasing has failed. Puts additional exceptions as supressed into throwed one, if multiple items have throwed. For failed release, initial exception may be provided in ex.

Mutates scope if it was an atom.

See also: grab-scope

scope-push!

Available since version 1.0 (view source)

Usage:
  • (scope-push! value-or-map & options)

Type signature:
  • (Any ⨯ Any) → {}

Pushes item or scope into current scope. Returns scope map. Item is a map with following keys:

  • :value - fn or releasable or weak reference to the releasable

  • :mode - optional - :exit/:success/:fail

  • :timeout - optional - nil (blocks)/IDuration

  • :timeout-mode - optional - nil (ignore)/:cancel/:throw

Scope defaults (see docs for with-scope) are used when some optional entries are not specified. If value-or-map is a scope, all its options, including those passed as additional args, are ignored. Mutates value-or-map if it was an atom.

See also: grab-scope, with-scope

with-io-scope

Available since version 1.0 (view source)

MACRO (with-io-scope & body)

Like with-scope, but additionally puts body inside io! block.

with-scope

Available since version 1.0 (view source)

MACRO (with-scope & body)

Executes body in a new scope. Returns result from evaluating body. Takes an optional existing scope map or scope atom or defaults map as first arg. Following keys are valid:

  • :pervasive - Boolean, defaults to nil

  • :timeout - nil (blocks, default)/IDuration

  • :timeout-mode - nil (ignore, default)/:cancel/:throw

Note if pervasive is enabled, user must manually handle waiting for child threads or else scoped items used in child threads gets released too early.

Mutates first arg if it is atom and there are more than one args. Automatically releases scope after body is evaluated, with :success mode, throwing if release fails. Throws when body throws, releasing scope with :fail mode.

System

In this section: assoc-deps deps start! system system?

assoc-deps

Available since version 1.0 (view source)

Usage:
  • (assoc-deps factory deps)

Type signature:
  • (Record ⨯ {}) → Record

Returns factory with deps associated as its dependency map.

See also: deps, system, start!

deps

Available since version 1.0 (view source)

Usage:
  • (deps x)

Type signature:
  • (Record) → {}

Returns a dependency map for a given object x.

See also: assoc-deps, system, start!

start!

Available since version 1.0 (view source)

Usage:
  • (start! system)

  • (start! system cfg)

  • (start! system cfg cast-fn)

Type signatures:
  • (ISystem) → {}

  • (ISystem ⨯ {}) → {}

  • (ISystem ⨯ {} ⨯ AnyFn) → {}

Starts the system and returns map of started components. If the system object is also an acquirable factory, performs acquire! and returns a resource object. Configures system and all its dependencies, if configurable, with cfg and cast-fn.

system

Available since version 1.0 (view source)

Usage:
  • (system & {:as keyvals})

Type signature:
  • (Any) → ISystem

Returns a system map.

See also: start!, deps, assoc-deps

system?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (system? x)

Type signature: Predicate

Returns true if object x satisfies ISystem protocol, false otherwise.

See also: ISystem, start!, acquire!, system

Other

control!

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (control! x k v)

  • (control! x k v & keyvals)

Type signatures:
  • (IControllable ⨯ Any ⨯ Any) → IAdjustable

  • (IControllable ⨯ Any ⨯ Any ⨯ (Va Any)) → IAdjustable

Adjusts the part of the controller and returns it (the controller for x).

controllable?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (controllable? x)

Type signature: Predicate

Returns true if object x satisfies IControllable protocol, false otherwise.

controller

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (controller x)

Type signature:
  • (IControllable) → IAdjustable

Returns a controller for x, a mutable reference to the map-like structure containing resource controls.

See also: control!, controllable?

flush!

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (flush! x)

Type signature:
  • (IFlushable) → nil

Flushes any pending operations and returns nil. May block.

See also: flushable?

flushable?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (flushable? x)

Type signature: Predicate

Returns true if object x satisfies IFlushable protocol, false otherwise.

See also: IFlushable, flush

position

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (position x)

Type signature:
  • (ISeekable) → IMutable

Returns mutable reference to the current position.

See also: seekable?, size

request!

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (request! x request)

Type signature:
  • (Any ⨯ Any) → Any

Returns a response to the request. Blocks.

experimental, subject to change.

See also: requestable?

seekable?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (seekable? x)

Type signature: Predicate

Returns true if object x satisfies ISeekable protocol, false otherwise.

See also: ISeekable, size, position

size

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (size x)

Type signature:
  • (ISeekable) → Integer+

Returns the current size of a seekable resource x, in resource specific units.

See also: position, seekable?

status

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (status x)

Type signature:
  • (IStatusable) → IReference

Returns an immutable reference to a the status map. Returned reference usually also behaves as a Mult.

See also: statusable?, status-of

status-of

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (status-of x k)

Type signature:
  • (IStatusable ⨯ Any) → Any

Returns current value of a status property under the key k for the object x.

See also: status, statusable?

statusable?

Available since version 1.0 (view source)

not referred automatically

Usage:
  • (statusable? x)

Type signature: Predicate

Returns true if object x satisfies IStatusable protocol, false otherwise.