AbstractChunga implements streams capable of chunked encoding on demand as defined in RFC 2616. For an example of how these streams can be used see Drakma.
The library needs a Common Lisp implementation that supports Gray streams and relies on David Lichteblau's trivial-gray-streams to offer portability between different Lisps.
Chunga is currently not optimized towards performance - it is rather intended to be easy to use and (if possible) to behave correctly.
The code comes with a BSD-style license so you can basically do with it whatever you want.
Download current version or visit the project on Github.
The esieast way to install Chunga is with Quicklisp
The current development version of Chunga can be found
MAKE-CHUNKED-STREAMwhich takes an open binary stream (called the underlying stream) as its single argument. A binary stream in this context means that if it's an input stream, you can apply
READ-SEQUENCEto it where the sequence is an array of element type
OCTET, and similarly for
WRITE-SEQUENCEand output streams. (Note that this specifically holds for bivalent streams like socket streams.)
A chunked stream behaves like an ordinary Lisp stream
with the addition that you can turn chunking on and off for
input as well as for output. With chunking turned on, data is read or
written according to
the definition in RFC
Every chunked stream returned by
MAKE-CHUNKED-STREAMis of this type which is a subtype of
A chunked stream is of this type if its underlying stream is an input stream. This is a subtype of
A chunked stream is of this type if its underlying stream is an output stream. This is a subtype of
A chunked stream is of this type if it is both a
CHUNKED-INPUT-STREAMas well as a
make-chunked-stream stream => chunked-stream
Creates and returns a chunked stream (a stream of type
CHUNKED-STREAM) which wraps
streammust be an open binary stream.
chunked-stream-stream (stream chunked-stream) => underlying-stream
Returns the underlying stream of the chunked stream
chunked-stream-input-chunking-p object => generalized-boolean
Returns a true value if
objectis of type
CHUNKED-INPUT-STREAMand if input chunking is currently enabled.
(setf (chunked-stream-input-chunking-p (stream chunked-input-stream)) new-value)
This function is used to switch input chunking on
streamon or off. Note that input chunking will usally be turned off automatically when the last chunk is read.
chunked-stream-output-chunking-p object => generalized-boolean
Returns a true value if
objectis of type
CHUNKED-OUTPUT-STREAMand if output chunking is currently enabled.
(setf (chunked-stream-output-chunking-p (stream chunked-output-stream)) new-value)
This function is used to switch output chunking on
streamon or off.
chunked-input-stream-extensions (stream chunked-input-stream) => extensions
Returns an alist of attribute/value pairs corresponding to the optional "chunk extensions" which might have been encountered when reading from
chunked-input-stream-trailers (stream chunked-input-stream) => trailers
Returns the optional "trailer" HTTP headers which might have been sent after the last chunk, i.e. directly before input chunking ended on
stream. The format of
trailersis identical to that returned by
All conditions signalled by Chunga are of this type. This is a subtype of
All errors signalled by Chunga are of this type. This is a subtype of
STREAM-ERROR-STREAMcan be used to access the offending stream.
All warnings signalled by Chunga are of this type. This is a subtype of
An error of this type is signalled if Chunga encounters wrong or unknown syntax when reading data. This is a subtype of
An error of this type is signalled if a function was called with inconsistent or illegal parameters. This is a subtype of
A condition of this type is signaled if an unexpected character (octet) is read while reading from a chunked stream with input chunking enabled. This is a subtype of
A condition of this type is signaled if we reach an unexpected EOF on a chunked stream with input chunking enabled. This is a subtype of
Note that all of these functions are designed to work
streams, specifically on streams with element
(UNSIGNED-BYTE 8). They will not work
with character streams. (But the "bivalent" streams offered by many
Lisp implementations will do.) They must be called within the context
with-character-stream-semantics statement* => result*
statement*forms in such a way that functions within this section can read characters from binary streams (treating octets as the Latin-1 characters with the corresponding code points). All the functions below must be wrapped with this macro. If your code uses several of these functions which interact on the same stream, all of them must be wrapped with the same macro. See the source code of Drakma or Hunchentoot for examples of how to use this macro.
read-line* stream &optional log-stream => line
Reads and assembles characters from the binary stream
streamuntil a carriage return is read. Makes sure that the following character is a linefeed. If
NIL, then the function will also accept a lone carriage return or linefeed as a line break. Returns the string of characters read excluding the line break. Additionally logs this string to
log-streamif it is not
read-http-headers stream &optional log-stream => headers
Reads HTTP header lines from the binary stream
stream(except for the initial status line which is supposed to be read already) and returns a corresponding alist of names and values where the names are keywords and the values are strings. Multiple lines with the same name are combined into one value, the individual values separated by commas. Header lines which are spread across multiple lines are recognized and treated correctly. (But see
*TREAT-SEMICOLON-AS-CONTINUATION*.) Additonally logs the header lines to
log-streamif it is not
read-token stream => token
Read characters from the binary stream
streamwhile they are token constituents (according to RFC 2616). It is assumed that there's a token character at the current position. The token read is returned as a string. Doesn't signal an error (but simply stops reading) if
END-OF-FILEis encountered after the first character.
token-char-p char => generalized-boolean
Returns a true value if the Lisp character
charis a token constituent according to RFC 2616.
read-name-value-pair stream &key value-required-p cookie-syntax => pair
Reads a typical (in RFC 2616) name/value or attribute/value combination from the binary stream
stream- a token followed by a
#\=character and another token or a quoted string. Returns a cons of the name and the value, both as strings. If
NIL(the default is
#\=sign and the value are optional. If
cookie-syntaxis true (the default is
NIL), the value is read like the value of a cookie header.
read-name-value-pairs stream &key value-required-p cookie-syntax => pairs
READ-NAME-VALUE-PAIRto read and return an alist of name/value pairs from the binary stream
stream. It is assumed that the pairs are separated by semicolons and that the first char read (except for whitespace) will be a semicolon. The parameters are used as in
READ-NAME-VALUE-PAIR. Stops reading in case of
END-OF-FILE(instead of signaling an error).
assert-char stream expected-char => char
Reads the next character from the binary stream
streamand checks if it is the character
expected-char. Signals an error otherwise.
skip-whitespace stream => char-or-nil
Consume characters from the binary stream
END-OF-FILEis encountered or a non-whitespace (according to RFC 2616) characters is seen. This character is returned (or
NILin case of
read-char* stream => char
Reads and returns the next character from the binary stream
peek-char* stream &optional eof-error-p eof-value => boolean
Returns a true value if a character can be read from the binary stream
eof-error-phas a true value, an error is signalled if no character remains to be read.
eof-valuespecifies the value to return if
eof-error-pis false and the end of the file has been reached.
trim-whitespace string &key start end => string'
Returns a version of the string
end) where spaces and tab characters are trimmed from the start and the end.
Used by the parsing functions in this section as an introduction to a standardized error message. Should be bound to a string or
NILif one of these functions is called.
Some web servers do not respond with a correct CRLF line ending for HTTP headers but with a lone linefeed or carriage return instead. If this variable is bound to a true value,
READ-LINE*will treat a lone LF or CR character as an acceptable end of line. The initial value is
According to John Foderaro, Netscape v3 web servers bogusly split
Set-Cookieheaders over multiple lines which means that we'd have to treat
Set-Cookieheaders ending with a semicolon as incomplete and combine them with the next header. This will only be done if this variable has a true value, though. Its default value is
as-keyword string &key destructivep => keyword
Converts the string
stringto a keyword where all characters are uppercase or lowercase, taking into account the current readtable case. Might destructively modify
destructivepis true which is the default. "Knows" several HTTP header names and methods and is optimized to not call
as-capitalized-string keyword => capitalized-string
Kind of the inverse of
AS-KEYWORD. Has essentially the same effect as
STRING-CAPITALIZEbut is optimized for "known" keywords like
Thanks to Jochen Schmidt's chunking code in ACL-COMPAT for inspiration. This documentation was prepared with DOCUMENTATION-TEMPLATE.