skip to navigation
skip to content

biomaj-process 3.0.10

BioMAJ process service

# About

Microservice to manage the process execution of biomaj.

A protobuf interface is available in biomaj_process/message/ to exchange messages between BioMAJ and the download service. Messages go through RabbitMQ (to be installed).

# Protobuf

To compile protobuf, in biomaj_process/message:

protoc –python_out=. message.proto

# Development

flake8 biomaj_process

# Run

## Message consumer: export BIOMAJ_CONFIG=path_to_config.yml python bin/

## Web server

If package is installed via pip, you need a file named containing somehwhere on local server:

def worker_exit(server, worker):
from prometheus_client import multiprocess multiprocess.mark_process_dead(

If you cloned the repository and installed it via python install, just refer to the in the cloned repository.

export BIOMAJ_CONFIG=path_to_config.yml rm -rf ..path_to/prometheus-multiproc mkdir -p ..path_to/prometheus-multiproc export prometheus_multiproc_dir=..path_to/prometheus-multiproc gunicorn -c ..path_to/ biomaj_download.biomaj_process_web:app

Web processes should be behind a proxy/load balancer, API base url /api/process

A prometheus /metrics endpoint is exposed on web server

README doc fixes Add dependency banks additional volumes for Docker
Allow local_endpoint definition per service Fix volumes for docker with dependency banks
Fix #1, DRMAA needs colon in front of output and error path (mandatory depending on versions)
fix case if process desc or type is missing in bank properties
disable web thread logging
retry in case of contact failure with proxy
for docker, restrict directory to bank directory and dependencies add host in prometheus stats
fix prometheus + gunicorn multi process add consul supervision
add logging info execute docker processes in biomaj micro service
bug fixes
move process management out of biomaj main package
File Type Py Version Uploaded on Size
biomaj_process-3.0.10-py2.py3-none-any.whl (md5) Python Wheel py2.py3 2017-12-18 16KB