A python helper library to report metrics and communicate with Polyaxon.
Project description
Polyaxon-helper
Polyaxon helper is a lightweight python library to report metrics and communicate information with Polyaxon.
Installation
$ pip install -U polyaxon-helper
for python3
$ pip3 install -U polyaxon-helper
Install polyaxon
Please check polyaxon installation guide
Install in polyaxonfile
If you want to delegate the installation to polyaxon during the build process,
add a new step to the run section in your polyaxonfile:
...
build:
image: ...
build_steps:
- ...
- pip install -U polyaxon-helper
- ...
run:
cmd: ...
Usage
Getting env variables defined by Polyaxon
from polyaxon_helper import (
get_cluster_def,
get_declarations,
get_experiment_info,
get_task_info,
get_tf_config,
get_job_info,
get_outputs_path,
get_outputs_refs_paths,
get_data_paths,
get_log_level
)
cluster_def = get_cluster_def()
declarations = get_declarations()
experiment_info = get_experiment_info()
task_info = get_task_info()
job_info = get_job_info()
outputs_path = get_outputs_path()
outputs_refs_paths = get_outputs_refs_paths()
data_paths = get_data_paths()
tf_config = get_tf_config()
log_level = get_log_level()
-
get_cluster_def: Returns cluster definition created by polyaxon.{ "master": ["plxjob-master0-8eefb7a1146f476ca66e3bee9b88c1de:2000"], "worker": ["plxjob-worker1-8eefb7a1146f476ca66e3bee9b88c1de:2000", "plxjob-worker2-8eefb7a1146f476ca66e3bee9b88c1de:2000"], "ps": ["plxjob-ps3-8eefb7a1146f476ca66e3bee9b88c1de:2000"], }
-
get_declarations: Returns all the experiment declarations based on both,- declarations section
- matrix section
-
get_tf_config: Returns the TF_CONFIG defining the cluster and the current task. ifenvvaris not null, it will set and env variable withenvvar. -
get_experiment_info: Returns information about the experiment.- project_name
- experiment_group_name
- experiment_name
- project_uuid
- experiment_group_uuid
- experiment_uuid
-
get_job_info: Returns information about the job.- project_name
- job_name
- project_uuid
- job_uuid
-
get_task_info: Returns the task info:{"type": str, "index": int}. -
outputs_path: The outputs path generated by polyaxon based on the hierarchy of the experiment.`user/project/group/experiment/files` -
data_paths: The data paths mounted for the job/experiment:{ 'data1': '/data/1', 'data-foo': '/data/foo', } -
outputs_path: The references outputs paths requested by the user, the order follows the order specified by the user:{ 'experiments': [ `user/project/job12/files`, ], 'experiments': [ `user/project/group/experiment1/files`, `user/project/experiment100/files` ] } -
get_log_level: If set on the polyaxonfile it will return the log level.
Reporting metrics to Polyaxon
In order to report metrics for an experiment, just add these lines in you program.
from polyaxon_helper import send_metrics
send_metrics(accuracy=0.9, precision=0.95)
Quick start
Please check our quick start guide to start training your first experiment.
License
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for polyaxon_helper-0.5.0-py2.py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 552934f42fb5fab24cd13148d262f8ed1fcfd340544d00c6c178e8bf7e018942 |
|
| MD5 | 3d2b19e99d073e4b848fa05ffe8de99a |
|
| BLAKE2b-256 | 36bd3cae6e3157f355f1f5db6ec7b0a55405147686026f0f7e23e4122948cf4b |
