Skip to main content

Facilities to do with buffers, particularly CornuCopyBuffer, an automatically refilling buffer to support parsing of data streams.

Project description

Facilities to do with buffers, particularly CornuCopyBuffer, an automatically refilling buffer to support parsing of data streams.

Function chunky(bfr_func)

Decorator for a function accepting a leading CornuCopyBuffer parameter. Returns a function accepting a leading data chunks parameter (bytes instances) and optional offset and 'copy_offsets` keywords parameters.

Example::

@chunky def func(bfr, ...):

Class CopyingIterator

Wrapper for an iterator that copies every item retrieved to a callable.

Class CornuCopyBuffer

An automatically refilling buffer intended to support parsing of data streams.

Attributes:

  • buf: a buffer of unparsed data from the input, available for direct inspection by parsers
  • offset: the logical offset of the buffer; this excludes unconsumed input data and .buf

The primary methods supporting parsing of data streams are extend() and take(). Calling .extend(min_size) arranges that .buf contains at least min_size bytes. Calling .take(size) fetches exactly size bytes from .buf and the input source if necessary and returns them, adjusting .buf.

len(CornuCopyBuffer) returns the length of .buf.

bool(CornuCopyBuffer) tests whether len() > 0.

Indexing a CornuCopyBuffer accesses .buf.

A CornuCopyBuffer is also iterable, yielding data in whatever sizes come from its input_data source, preceeded by the current .buf if not empty.

A CornuCopyBuffer also supports the file methods .read, .tell and .seek supporting drop in use of the buffer in many file contexts. Backward seeks are not supported. .seek will take advantage of the input_data's .seek method if it has one, otherwise it will use reads.

Class SeekableFDIterator

MRO: SeekableIterator
An iterator over the data of a seekable file descriptor.

Note: the iterator works with an os.dup() of the file descriptor so that it can close it with impunity; this requires the caller to close their descriptor.

Class SeekableFileIterator

MRO: SeekableIterator
An iterator over the data of a file object.

Note: the iterator closes the file on del or if its .close method is called.

Class SeekableIterator

A base class for iterators over seekable things.

Class SeekableMMapIterator

MRO: SeekableIterator
An iterator over the data of a mappable file descriptor.

Note: the iterator works with an mmap of an os.dup() of the file descriptor so that it can close it with impunity; this requires the caller to close their descriptor.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cs.buffer-20180810.tar.gz (10.4 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page