Skip to main content

Flex is a framework for building and executing computing pipelines

Project description

Muscle for data science

Data science projects get disorganized quickly. Every test involves a new script, each script requires a panoply of arguments and produces one or more data files. Keeping track of all this implied structure is a pain - what is this script for? What does it depend on? What created this data file? Which parameters updated this table in the database?

Enter flex - a utility that allows you to express and run all the computational tasks in a project. Crucially, it captures the specific parameters used for each task, the data files produced, and any dependencies that task has on other tasks. All this is captured in files called pipelines (which can even be connected to one another). Toss in some helpful comments, and you have executable documentation for your project.

It aims to achieve all this without introducing any overhead. You won’t have to write more code than you currently are doing, write or maintain any extra documentation, or use fancy data management solutions. Whatever you are already doing, flex is compatible with it.

Project details


Release history Release notifications | RSS feed

This version

1.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flexds-1.1.tar.gz (18.5 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page