Skip to main content

Neuromation Platform API client

Project description

codecov

Table of Contents

Preface

Welcome to Neuromation API Python client. Package ship command line tool called neuro. With neuro you can:

neuro

Usage:

neuro [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
-v, --verbose Enable verbose mode.
--show-traceback Show python traceback on error, useful for debugging the tool.
--color [yes | no | auto] Color mode.
--disable-pypi-version-check Don't periodically check PyPI to determine whether a new version of Neuromation CLI is available for download.
--version Show the version and exit.
--help Show this message and exit.

Command Groups:

Usage Description
neuro job Job operations
neuro storage Storage operations
neuro image Container image operations
neuro config Client configuration
neuro completion Output shell completion code

Commands:

Usage Description
neuro help Get help on a command
neuro submit Submit an image to run on the cluster
neuro ps List all jobs
neuro status Display status of a job
neuro exec Execute command in a running job
neuro port-forward Forward a port of a running job exposed with -ssh option to a local port
neuro logs Print the logs for a container
neuro kill Kill job(s)
neuro top Display GPU/CPU/Memory usage
neuro login Log into Neuromation Platform
neuro logout Log out
neuro cp Copy files and directories
neuro ls List directory contents
neuro rm Remove files or directories
neuro mkdir Make directories
neuro mv Move or rename files and directories
neuro images List images
neuro push Push an image to platform registry
neuro pull Pull an image from platform registry
neuro share Shares resource specified by URI to a USER with PERMISSION Examples: neuro share...

neuro job

Job operations.

Usage:

neuro job [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro job submit Submit an image to run on the cluster
neuro job ls List all jobs
neuro job status Display status of a job
neuro job exec Execute command in a running job
neuro job port-forward Forward a port of a running job exposed with -ssh option to a local port
neuro job logs Print the logs for a container
neuro job kill Kill job(s)
neuro job top Display GPU/CPU/Memory usage

neuro job submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro job submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro job submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container pytorch:latest with connection enabled to port 22 and
# sets PYTHONPATH environment value to /python.
# Please note that SSH server should be provided by container.
neuro job submit --env PYTHONPATH=/python --volume storage:/data/2018q1:/data:ro --ssh 22 pytorch:latest

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http INTEGER Enable HTTP port forwarding to container
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
--ssh INTEGER Enable SSH port forwarding to container
-p, --preemptible / -P, --non-preemptible Run job on a lower-cost preemptible instance [default: True]
-d, --description DESC Add optional description to the job
-q, --quiet Run command in quiet mode (print only job id)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--help Show this message and exit.

neuro job ls

List all jobs.

Usage:

neuro job ls [OPTIONS]

Examples:

neuro job ls --description=my favourite job
neuro job ls --status=all
neuro job ls -s pending -s running -q

Options:

Name Description
-s, --status [pending | running | succeeded | failed | all] Filter out job by status (multiple option)
-d, --description DESCRIPTION Filter out job by job description (exact match)
-q, --quiet Print only Job ID
-w, --wide Do not cut long lines for terminal width
--help Show this message and exit.

neuro job status

Display status of a job.

Usage:

neuro job status [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro job exec

Execute command in a running job.

Usage:

neuro job exec [OPTIONS] ID CMD...

Options:

Name Description
-t, --tty Allocate virtual tty. Useful for interactive jobs.
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro job port-forward

Forward a port of a running job exposed with -ssh option to a local port.

Usage:

neuro job port-forward [OPTIONS] ID LOCAL_PORT REMOTE_PORT

Options:

Name Description
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro job logs

Print the logs for a container.

Usage:

neuro job logs [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro job kill

Kill job(s).

Usage:

neuro job kill [OPTIONS] ID...

Options:

Name Description
--help Show this message and exit.

neuro job top

Display GPU/CPU/Memory usage.

Usage:

neuro job top [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro storage

Storage operations.

Usage:

neuro storage [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro storage cp Copy files and directories
neuro storage ls List directory contents
neuro storage rm Remove files or directories
neuro storage mkdir Make directories
neuro storage mv Move or rename files and directories

neuro storage cp

Copy files and directories.

Either SOURCE or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Usage:

neuro storage cp [OPTIONS] SOURCE DESTINATION

Examples:

# copy local file ./foo into remote storage root
neuro storage cp ./foo storage:///
neuro storage cp ./foo storage:/

# download remote file foo into local file foo with
# explicit file:// scheme set
neuro storage cp storage:///foo file:///foo

Options:

Name Description
-r, --recursive Recursive copy, off by default
-p, --progress Show progress, off by default
--help Show this message and exit.

neuro storage ls

List directory contents.

By default PATH is equal user`s home dir (storage:)

Usage:

neuro storage ls [OPTIONS] [PATH]

Options:

Name Description
-h, --human-readable with -l print human readable sizes (e.g., 2K, 540M)
-l use a long listing format
--sort [name | size | time] sort by given field, default is name
--help Show this message and exit.

neuro storage rm

Remove files or directories.

Usage:

neuro storage rm [OPTIONS] PATH

Examples:

neuro storage rm storage:///foo/bar/
neuro storage rm storage:/foo/bar/
neuro storage rm storage://{username}/foo/bar/

Options:

Name Description
--help Show this message and exit.

neuro storage mkdir

Make directories.

Usage:

neuro storage mkdir [OPTIONS] PATH

Options:

Name Description
--help Show this message and exit.

neuro storage mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro storage mv [OPTIONS] SOURCE DESTINATION

Examples:

# move or rename remote file
neuro storage mv storage://{username}/foo.txt storage://{username}/bar.txt
neuro storage mv storage://{username}/foo.txt storage://~/bar/baz/foo.txt

# move or rename remote directory
neuro storage mv storage://{username}/foo/ storage://{username}/bar/
neuro storage mv storage://{username}/foo/ storage://{username}/bar/baz/foo/

Options:

Name Description
--help Show this message and exit.

neuro image

Container image operations.

Usage:

neuro image [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro image ls List images
neuro image push Push an image to platform registry
neuro image pull Pull an image from platform registry

neuro image ls

List images.

Usage:

neuro image ls [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro image push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contains tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro image push [OPTIONS] IMAGE_NAME [REMOTE_IMAGE_NAME]

Examples:

neuro image push myimage
neuro image push alpine:latest image:my-alpine:production
neuro image push alpine image://myfriend/alpine:shared

Options:

Name Description
--help Show this message and exit.

neuro image pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro image pull [OPTIONS] IMAGE_NAME [LOCAL_IMAGE_NAME]

Examples:

neuro image pull image:myimage
neuro image pull image://myfriend/alpine:shared
neuro image pull image://username/my-alpine:production alpine:from-registry

Options:

Name Description
--help Show this message and exit.

neuro config

Client configuration.

Usage:

neuro config [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro config login Log into Neuromation Platform
neuro config show Print current settings
neuro config show-token Print current authorization token
neuro config auth Update authorization token
neuro config logout Log out

neuro config login

Log into Neuromation Platform.

Usage:

neuro config login [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro config show

Print current settings.

Usage:

neuro config show [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config show-token

Print current authorization token.

Usage:

neuro config show-token [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config auth

Update authorization token.

Usage:

neuro config auth [OPTIONS] TOKEN

Options:

Name Description
--help Show this message and exit.

neuro config logout

Log out.

Usage:

neuro config logout [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro completion

Output shell completion code.

Usage:

neuro completion [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro completion generate Provide an instruction for shell completion generation
neuro completion patch Automatically patch shell configuration profile to enable completion

neuro completion generate

Provide an instruction for shell completion generation.

Usage:

neuro completion generate [OPTIONS] [bash|zsh]

Options:

Name Description
--help Show this message and exit.

neuro completion patch

Automatically patch shell configuration profile to enable completion

Usage:

neuro completion patch [OPTIONS] [bash|zsh]

Options:

Name Description
--help Show this message and exit.

neuro help

Get help on a command.

Usage:

neuro help [OPTIONS] [COMMAND]...

Options:

Name Description
--help Show this message and exit.

neuro submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro job submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container pytorch:latest with connection enabled to port 22 and
# sets PYTHONPATH environment value to /python.
# Please note that SSH server should be provided by container.
neuro job submit --env PYTHONPATH=/python --volume storage:/data/2018q1:/data:ro --ssh 22 pytorch:latest

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http INTEGER Enable HTTP port forwarding to container
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
--ssh INTEGER Enable SSH port forwarding to container
-p, --preemptible / -P, --non-preemptible Run job on a lower-cost preemptible instance [default: True]
-d, --description DESC Add optional description to the job
-q, --quiet Run command in quiet mode (print only job id)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--help Show this message and exit.

neuro ps

List all jobs.

Usage:

neuro ps [OPTIONS]

Examples:

neuro job ls --description=my favourite job
neuro job ls --status=all
neuro job ls -s pending -s running -q

Options:

Name Description
-s, --status [pending | running | succeeded | failed | all] Filter out job by status (multiple option)
-d, --description DESCRIPTION Filter out job by job description (exact match)
-q, --quiet Print only Job ID
-w, --wide Do not cut long lines for terminal width
--help Show this message and exit.

neuro status

Display status of a job.

Usage:

neuro status [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro exec

Execute command in a running job.

Usage:

neuro exec [OPTIONS] ID CMD...

Options:

Name Description
-t, --tty Allocate virtual tty. Useful for interactive jobs.
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro port-forward

Forward a port of a running job exposed with -ssh option to a local port.

Usage:

neuro port-forward [OPTIONS] ID LOCAL_PORT REMOTE_PORT

Options:

Name Description
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro logs

Print the logs for a container.

Usage:

neuro logs [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro kill

Kill job(s).

Usage:

neuro kill [OPTIONS] ID...

Options:

Name Description
--help Show this message and exit.

neuro top

Display GPU/CPU/Memory usage.

Usage:

neuro top [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro login

Log into Neuromation Platform.

Usage:

neuro login [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro logout

Log out.

Usage:

neuro logout [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro cp

Copy files and directories.

Either SOURCE or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Usage:

neuro cp [OPTIONS] SOURCE DESTINATION

Examples:

# copy local file ./foo into remote storage root
neuro storage cp ./foo storage:///
neuro storage cp ./foo storage:/

# download remote file foo into local file foo with
# explicit file:// scheme set
neuro storage cp storage:///foo file:///foo

Options:

Name Description
-r, --recursive Recursive copy, off by default
-p, --progress Show progress, off by default
--help Show this message and exit.

neuro ls

List directory contents.

By default PATH is equal user`s home dir (storage:)

Usage:

neuro ls [OPTIONS] [PATH]

Options:

Name Description
-h, --human-readable with -l print human readable sizes (e.g., 2K, 540M)
-l use a long listing format
--sort [name | size | time] sort by given field, default is name
--help Show this message and exit.

neuro rm

Remove files or directories.

Usage:

neuro rm [OPTIONS] PATH

Examples:

neuro storage rm storage:///foo/bar/
neuro storage rm storage:/foo/bar/
neuro storage rm storage://{username}/foo/bar/

Options:

Name Description
--help Show this message and exit.

neuro mkdir

Make directories.

Usage:

neuro mkdir [OPTIONS] PATH

Options:

Name Description
--help Show this message and exit.

neuro mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro mv [OPTIONS] SOURCE DESTINATION

Examples:

# move or rename remote file
neuro storage mv storage://{username}/foo.txt storage://{username}/bar.txt
neuro storage mv storage://{username}/foo.txt storage://~/bar/baz/foo.txt

# move or rename remote directory
neuro storage mv storage://{username}/foo/ storage://{username}/bar/
neuro storage mv storage://{username}/foo/ storage://{username}/bar/baz/foo/

Options:

Name Description
--help Show this message and exit.

neuro images

List images.

Usage:

neuro images [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contains tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro push [OPTIONS] IMAGE_NAME [REMOTE_IMAGE_NAME]

Examples:

neuro image push myimage
neuro image push alpine:latest image:my-alpine:production
neuro image push alpine image://myfriend/alpine:shared

Options:

Name Description
--help Show this message and exit.

neuro pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro pull [OPTIONS] IMAGE_NAME [LOCAL_IMAGE_NAME]

Examples:

neuro image pull image:myimage
neuro image pull image://myfriend/alpine:shared
neuro image pull image://username/my-alpine:production alpine:from-registry

Options:

Name Description
--help Show this message and exit.

neuro share

Shares resource specified by URI to a USER with PERMISSION

Usage:

neuro share [OPTIONS] URI USER [read|write|manage]

Examples:

neuro share storage:///sample_data/ alice manage
neuro share image:resnet50 bob read
neuro share job:///my_job_id alice write

Options:

Name Description
--help Show this message and exit.

Api

TODO

Contributing

git clone https://github.com/neuromation/platform-api-clients.git
cd platform-api-clients/python

Before you begin, it is recommended to have clean virtual environment installed:

python -m venv .env
source .env/bin/activate

Development flow:

  • Install dependencies: make init
  • Run tests: make test
  • Lint: make lint
  • Publish to pypi: make publish

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

neuromation-0.7.0-py3-none-any.whl (64.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page