Python script to submit torque jobs in a very hackish way
Project description
This is as very simple hackish hack, that allows you to any serializable python callable as a torque job.
Features:
Allows you to send enviorment on the other side
Allows to execute any callable function as a torque task
Allows you to launch a bunch of taska as a torque array job.
Dependencies:
Working python 2.7/3.4 enviorment
Some python dependencies (see REQUIREMENTS)
Working qsub command (no need for other PBS/torque stuff).
Works by serializing the enviorment and callable function to the enviorment variables (or enviorment variables and files). For serialization we use dill if avilable or pickle.
Enviorment can be initialized using arbirtary bash script — this script will be sourced before running provided python callable.
Example
Example without enviorment:
callable = partial(print, "Hello World!")
from torqsubmit import Submitter
s = Submitter()
s.tasks = [callable]
s.submit()Example with enviorment:
callable = partial(print, "Hello World!")
from torqsubmit._submit import Submitter
ENV = """
source ${HOME}/.bashrc
workon torque-submit
export MSG="Hello World!"
"""
def print_from_env():
import os
print(os.environ["MSG"])
s = Submitter()
s.tasks = [callable]
s.enviorment = ENV
s.submit()Example submitting many tasks:
from __future__ import print_function
from functools import partial
callable = partial(print, "Hello World!")
from torqsubmit._submit import Submitter
ENV = """
source ${HOME}/.bashrc
workon torque-submit
export MSG="Hello World!"
"""
def print_from_env():
import os
print(os.environ["MSG"])
s = Submitter()
s.enviorment = ENV
s.tasks = [print_from_env, print_from_env]
s.submit()Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
