skip to navigation
skip to content

biomaj-download 3.0.14

BioMAJ download service

# About

Microservice to manage the downloads of biomaj.

A protobuf interface is available in biomaj_download/message/ to exchange messages between BioMAJ and the download service. Messages go through RabbitMQ (to be installed).

# Protobuf

To compile protobuf, in biomaj_download/message:

protoc –python_out=. message.proto

# Development

flake8 biomaj_download/*.py biomaj_download/download

# Run

## Message consumer:

export BIOMAJ_CONFIG=path_to_config.yml python bin/

## Web server

If package is installed via pip, you need a file named containing somehwhere on local server:

def worker_exit(server, worker):
from prometheus_client import multiprocess multiprocess.mark_process_dead(

If you cloned the repository and installed it via python install, just refer to the in the cloned repository.

export BIOMAJ_CONFIG=path_to_config.yml rm -rf ..path_to/prometheus-multiproc mkdir -p ..path_to/prometheus-multiproc export prometheus_multiproc_dir=..path_to/prometheus-multiproc gunicorn -c biomaj_download.biomaj_download_web:app

Web processes should be behind a proxy/load balancer, API base url /api/download

Allow setup of local_endpoint per service, else use default local_endpoint
In rate limiting, add progress vs total of download Fix rate limiting submission
Add retry in case of session creation failure disable web thread logging
Display progress of download by percent of downloads In case of contact error in downloadclient, retry connection
Feature #3: Add rate limiting option to limit number of parallel downloads for a client
Add host in prometheus stats Fix #2: allow setting or to -1 if not avalaible in http(s) page for regexp
Fix prometheus stats Add consul supervision
Change size type to int64
Fix download_or_copy to avoid downloading a file existing in a previous production directory
Fixes on messages
Fix management of timeout leading to a crash when using parameter.
set rabbitmq parameter optional
add missing README etc.. in package
move download management out of biomaj main package
File Type Py Version Uploaded on Size
biomaj_download-3.0.14-py2.py3-none-any.whl (md5) Python Wheel py2.py3 2017-12-08 36KB