git clone 'https://github.com/rpav/fast-io.git'
Now with static-vectors support!
(deftype octet '(unsigned-byte 8)) (deftype octet-vector '(simple-array octet (*)))
Fast-io is about improving performance to octet-vectors and octet streams (though primarily the former, while wrapping the latter). Imagine we're creating messages for the network. If we try and fill an octet-vector with 50 bytes, 50000 times, here are the results (SBCL 1.0.57):
t/benchmarks.lisp for the exact code used.)
It should be surprising that it takes a nontrivial effort to achieve relatively decent performance to octet-vectors, but probably isn't. However, fast-io provides a relatively straightforward interface for reading and writing either a stream or a vector:
;;; Write a byte or sequence, optionally to a stream: (with-fast-output (buffer [STREAM | :vector | :static]) (fast-write-byte BYTE buffer)) (with-fast-output (buffer [STREAM | :vector | :static]) (fast-write-sequence OCTET-VECTOR buffer [START [END]])) ;;; Read from a vector or stream: (with-fast-input (buffer VECTOR [STREAM]) (fast-read-byte buffer)) (with-fast-input (buffer VECTOR [STREAM]) (let ((vec (make-octet-vector N))) (fast-read-sequence vec buffer [START [END]])))
Fast-io provides a host of read and write functions for big- and little-endian reads. See the Dictionary below.
You may now specify
:static instead of a stream to
WITH-OUTPUT-BUFFER. This returns an octet-vector created with
which means that passing the buffered data directly to a foreign
function is now that much more efficient:
(let ((data (with-fast-output (buffer :static) (buffer-some-data buffer)))) (foreign-send (static-vectors:static-vector-pointer data)) (static-vectors:free-static-vector data))
Note that the restriction for manually freeing the result remains. This avoids multiple inefficient (i.e., byte-by-byte) copies to foreign memory.
Obviously, the above API isn't built around Lisp streams, or even
gray-streams. However, fast-io provides a small wrapper using
trivial-gray-streams, and supports
(let ((stream (make-instance 'fast-io:fast-output-stream))) (write-sequence (fast-io:octets-from '(1 2 3 4)) stream))
fast-output-stream support backing a
stream, much like using the plain fast-io buffers. However, using the
gray-streams interface is a 3-4x as slow as using the buffers alone.
Simple benchmarks show the gray-streams interface writing 1M 50-byte
vectors in about 1.7s, whereas simply using buffers is about 0.8s.
Consing remains similar between the two.
Most functions operate on or require octet-vectors, i.e.,
(deftype octet () '(unsigned-byte 8)) (deftype octet-vector '(simple-array octet (*)))
Which is exactly what is defined and exported from
make-input-buffer &key VECTOR STREAM POS
Create an input buffer for use with input functions.
:vector specifies the vector to be read from.
:stream specifies the stream to read from.
:pos specifies the offset to start reading into
make-output-buffer &key OUTPUT
Create an output buffer for use with output functions.
:output specifies an output stream. If
:output :static is specified, and static-vectors is supported, output will be to a static-vector.
Finish the output and return the complete octet-vector.
Return the current read/write position for
with-fast-input (BUFFER VECTOR &optional STREAM (OFFSET 0)) &body body
Create an input buffer called
BUFFER, optionally reading from
VECTOR, followed by reading from
OFFSET is specified, start reading from this position in
with-fast-output (BUFFER &optional OUTPUT) &body BODY
Create an output buffer named
BUFFER, optionally writing to the stream
OUTPUT. This will automatically
BUFFER. Thus the
with-fast-output form evaluates to the completed octet-vector.
fast-read-byte INPUT-BUFFER &optional (EOF-ERROR-P t) EOF-VALUE
t, reading past the end-of-file will signal
CL:END-OF-FILE. Otherwise, it will return
fast-write-byte BYTE OUTPUT-BUFFER
fast-read-sequence SEQUENCE INPUT-BUFFER &optional (START 0) END
SEQUENCE. Values will be written starting at position
ENDis specified, ending at
END. Otherwise values will be written until the length of the sequence, or until the input is exhausted.
fast-write-sequence SEQUENCE OUTPUT-BUFFER &optional (START 0) END
OUTPUT-BUFFER, starting at position
ENDis specified, values will be written until
END; otherwise, values will be written for the length of the sequence.
For multi-byte reads and writes requiring endianness, fast-io provides functions in the following forms:
(write32-be VALUE BUFFER)will write the specified 32-bit value to the specified buffer with a big-endian layout. Likewise,
(writeu16-le VALUE BUFFER)will write an unsigned 16-bit value in little-endian layout.
(read64-le BUFFER)will read a 64-bit value from the buffer with little-endian layout.