Skip to main content

Simple statefile-driven task schedulers for AiiDA

Project description

Build Status Coverage Status Docs status PyPI version

aiida-statefile-schedulers

Simple statefile-driven task schedulers: Currently AiiDA relies mostly on full fledged task schedulers to run jobs in complex workflows. Running such workflows with the direct scheduler often means that many processes run together even when running the workflow directly (e.g. not submitting to the daemon), overloading a single node.

This scheduler does not run any jobs. Instead, it creates state files of the form $jobid.QUEUED in the directory ${AIIDA_STATE_DIR} (an environment variable you have to set in your .profile/.bash_profile of the target machine), waiting for some script to pick the jobs up and run them.

This runner script should create a file $jobid.DONE when done. As intermediate steps it can also create a file $jobid.RUNNING to signal AiiDA that it picked up a job. The initial state file contains lines of the form key=value with the following keys:

  • cwd: the working directory for this job
  • cmd: the command to run there (usually via bash -e ...)

The state files can also be renamed instead of created. A sample runner can be found in scripts/.

Features

Installation

pip install aiida-statefile-scheduler
verdi quicksetup  # better to set up a new profile
verdi plugin list aiida.schedulers  # should now show your calculation plugins

Development

git clone https://github.com/dev-zero/aiida-statefile-schedulers .
cd aiida-statefile-schedulers
pip install -e .[pre-commit,testing]  # install extra dependencies
pre-commit install  # install pre-commit hooks
pytest -v  # discover and run all tests

See the developer guide for more information.

License

MIT

Contact

tm@dev-zero.ch

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aiida-statefile-schedulers-0.2.1.tar.gz (3.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page