A library for transparent transformation of indexable containers (lists, etc.)
Project description
SeqTools
SeqTools extends the functionalities of itertools to indexable (list-like) objects. Some of the provided functionalities include: element-wise function mapping, reordering, reindexing, concatenation, joining, slicing, minibatching, etc.
SeqTools functions implement on-demand evaluation under the hood: operations and transformations are only applied to individual items when they are actually accessed. A simple but powerful prefetch function is also provided to quickly evaluate elements.
SeqTools originally targets data science, more precisely the data preprocessing stages. Being aware of the experimental nature of this usage, on-demand execution is made as transparent as possible by providing fault-tolerant functions and insightful error message.
Example
Example
>>> def f1(x): ... return x + 1 ... >>> def f2(x): # slow and memory heavy transformation ... time.sleep(.01) ... return [x for _ in range(500)] ... >>> def f3(x): ... return sum(x) / len(x) ... >>> data = list(range(1000))
Without seqtools, defining the pipeline and reading values looks like so:
>>> tmp1 = [f1(x) for x in data] >>> tmp2 = [f2(x) for x in tmp1] # takes 10 seconds and a lot of memory >>> res = [f3(x) for x in tmp2] >>> print(res[2]) 3.0 >>> print(max(tmp2[2])) # requires to store 499 500 useless values along 3
With seqtools:
>>> tmp1 = seqtools.smap(f1, data) >>> tmp2 = seqtools.smap(f2, tmp1) >>> res = seqtools.smap(f3, tmp2) # no computations so far >>> print(res[2]) # takes 0.01 seconds 3.0 >>> print(max(tmp2[2])) # easy access to intermediate results 3
Batteries included!
The library comes with a set of functions to manipulate sequences:
and others (suggestions are also welcome).
Installation
pip install seqtools
Documentation
The documentation is hosted at https://seqtools-doc.readthedocs.io.
Contributing and Support
Use the issue tracker to request features, propose improvements or report issues. For questions regarding usage, please send an email.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for SeqTools-1.2.0-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b74cdbb3f23d91f61c97d13371293e43861fe5f4e06c42b900a5e053f6a4199 |
|
MD5 | eddefdc1abba6eeedb40699b064f40e9 |
|
BLAKE2b-256 | 8ee5022aa6738d497a6704794353d2752f7449887e1eb843aa77ded4d6fcc657 |
Hashes for SeqTools-1.2.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a5ca97ac5065f131ec4c5eeb24d934a189de71d275f9b780aa0a581687cfd07f |
|
MD5 | 7f848bc0f2b08cf9934a444e6bdb8b48 |
|
BLAKE2b-256 | 499ea0a6c63a3f9cb2e8915cb4c067412f62d0f8783e4bc0346548e0aefae08c |
Hashes for SeqTools-1.2.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0a607d4fc286b2adcc41b02205efafa28a0429186ba44feb83bd35d98ab2976c |
|
MD5 | ef23f21743fb43ea113f9d3e4634a6dc |
|
BLAKE2b-256 | 22f17b0dabfcf8aadfbe7cb7f5f14ea4d215a81b39db8955eb8397952327a018 |
Hashes for SeqTools-1.2.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0eacc46397fc00cf14b99d73af4d29c7251abe26004a4569ee86789d3fb2fc32 |
|
MD5 | bcd6cec625c044e0a11a80151572353e |
|
BLAKE2b-256 | 01af75923d6bf2978bc08ef11599b739cbcab31f287d29c82ab668f005d1183e |
Hashes for SeqTools-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 92e606a3962cc92f69a9b968ade3633cef78277e3e73fd435e27fba659f25c9a |
|
MD5 | d9170041bc1567afd913fd03755bfa95 |
|
BLAKE2b-256 | 84c99f5d12ca8cd63df172d71de2401d9ce1afb46f844f7f24fcc8de04c0b002 |
Hashes for SeqTools-1.2.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3516453fa6430482fb90ca731a9a8d64b8467d232c2e0964026b5b5411c56c3 |
|
MD5 | b01589dbaf0210bfb4632f0c72694aef |
|
BLAKE2b-256 | a2530a7d4e3e7771efffc80983313606fb69d3074ba08df8492c564accbd394c |