Skip to main content

A library for transparent transformation of indexable containers (lists, etc.)

Project description

PyPi package Documentation

SeqTools

SeqTools extends the functionalities of itertools to indexable (list-like) objects. Some of the provided functionalities include: element-wise function mapping, reordering, reindexing, concatenation, joining, slicing, minibatching, etc.

SeqTools functions implement on-demand evaluation under the hood: operations and transformations are only applied to individual items when they are actually accessed. A simple but powerful prefetch function is also provided to eagerly evaluate elements in background threads or processes.

SeqTools originally targets data science, more precisely the data preprocessing stages. Being aware of the experimental nature of this usage, on-demand execution is made as transparent as possible by providing fault-tolerant functions and insightful error message.

Example

>>> def count_lines(filename):
...     with open(filename) as f:
...         return len(f.readlines())
>>>
>>> def count_words(filename):
...     with open(filename) as f:
...         return len(f.read().split())
>>>
>>> filenames = ["a.txt", "b.txt", "c.txt", "d.txt"]
>>> lc = seqtools.smap(count_lines, filenames)
>>> wc = seqtools.smap(count_words, filenames)
>>> counts = seqtools.collate([lc, wc])
>>> # no computations so far!
>>> lc[2]  # only evaluates on index 2
3
>>> counts[1]  # same for index 1
(1, 2)

Batteries included!

The library comes with a set of functions to manipulate sequences:

concatenate

concatenate

batch

batch

gather

gather

prefetch

prefetch

interleave

interleave

uniter

uniter

and others (suggestions are also welcome).

Installation

pip install seqtools

Documentation

The documentation is hosted at https://seqtools-doc.readthedocs.io.

Contributing and Support

Use the issue tracker to request features, propose improvements or report issues. For questions regarding usage, please send an email.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

SeqTools-1.4.0.tar.gz (595.9 kB view hashes)

Uploaded Source

Built Distributions

SeqTools-1.4.0-cp311-cp311-win_amd64.whl (37.3 kB view hashes)

Uploaded CPython 3.11 Windows x86-64

SeqTools-1.4.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (44.7 kB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

SeqTools-1.4.0-cp311-cp311-macosx_10_9_x86_64.whl (34.0 kB view hashes)

Uploaded CPython 3.11 macOS 10.9+ x86-64

SeqTools-1.4.0-cp310-cp310-win_amd64.whl (37.3 kB view hashes)

Uploaded CPython 3.10 Windows x86-64

SeqTools-1.4.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (44.6 kB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

SeqTools-1.4.0-cp310-cp310-macosx_10_9_x86_64.whl (34.0 kB view hashes)

Uploaded CPython 3.10 macOS 10.9+ x86-64

SeqTools-1.4.0-cp39-cp39-win_amd64.whl (37.3 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

SeqTools-1.4.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (44.4 kB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

SeqTools-1.4.0-cp39-cp39-macosx_10_9_x86_64.whl (34.0 kB view hashes)

Uploaded CPython 3.9 macOS 10.9+ x86-64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page