Skip to main content

Neuromation Platform API client

Project description

codecov

Table of Contents

Preface

Welcome to Neuromation API Python client. Package ship command line tool called neuro. With neuro you can:

neuro

Usage:

neuro [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
-v, --verbose Give more output. Option is additive, and can be used up to 2 times.
-q, --quiet Give less output. Option is additive, and can be used up to 2 times.
--neuromation-config PATH Path to config file.
--show-traceback Show python traceback on error, useful for debugging the tool.
--color [yes | no | auto] Color mode.
--disable-pypi-version-check Don't periodically check PyPI to determine whether a new version of Neuromation CLI is available for download.
--network-timeout FLOAT Network read timeout, seconds.
--version Show the version and exit.
--help Show this message and exit.

Command Groups:

Usage Description
neuro job Job operations
neuro storage Storage operations
neuro image Container image operations
neuro config Client configuration
neuro completion Output shell completion code
neuro acl Access Control List management

Commands:

Usage Description
neuro help Get help on a command
neuro run Run a job with predefined resources configuration
neuro submit Submit an image to run on the cluster
neuro ps List all jobs
neuro status Display status of a job
neuro exec Execute command in a running job
neuro port-forward Forward port(s) of a running job to local port(s)
neuro logs Print the logs for a container
neuro kill Kill job(s)
neuro top Display GPU/CPU/Memory usage
neuro save Save job's state to an image Examples: neuro job save job-id...
neuro login Log into Neuromation Platform
neuro logout Log out
neuro cp Copy files and directories
neuro ls List directory contents
neuro rm Remove files or directories
neuro mkdir Make directories
neuro mv Move or rename files and directories
neuro images List images
neuro push Push an image to platform registry
neuro pull Pull an image from platform registry
neuro share Shares resource specified by URI to a USER with PERMISSION Examples: neuro acl...

neuro job

Job operations.

Usage:

neuro job [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro job run Run a job with predefined resources configuration
neuro job submit Submit an image to run on the cluster
neuro job ls List all jobs
neuro job status Display status of a job
neuro job exec Execute command in a running job
neuro job port-forward Forward port(s) of a running job to local port(s)
neuro job logs Print the logs for a container
neuro job kill Kill job(s)
neuro job top Display GPU/CPU/Memory usage
neuro job save Save job's state to an image Examples: neuro job save job-id...
neuro job browse Opens a job's URL in a web browser

neuro job run

Run a job with predefined resources configuration.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro job run [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest on a machine with smaller GPU resources
# (see exact values in `neuro config show`) and with two volumes mounted:
#   storage://<home-directory>   --> /var/storage/home (in read-write mode),
#   storage://neuromation/public --> /var/storage/neuromation (in read-only mode).
neuro run --preset=gpu-small --volume=HOME pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 and arg2 as its arguments:
neuro run -s cpu-small image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2

Options:

Name Description
-s, --preset PRESET Predefined resource configuration (to see available values, run neuro config show)
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container [default: 80]
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-n, --name NAME Optional job name
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage://~:/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
--help Show this message and exit.

neuro job submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro job submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 arg2 arg3 as its arguments:
neuro submit image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2 arg3

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
--tpu-type TYPE TPU type to use
--tpu-sw-version VERSION Requested TPU software version
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-p, --preemptible / -P, --non-preemptible Run job on a lower-cost preemptible instance [default: False]
-n, --name NAME Optional job name
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage://~:/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
--help Show this message and exit.

neuro job ls

List all jobs.

Usage:

neuro job ls [OPTIONS]

Examples:

neuro ps -a
neuro ps -a --owner=user-1 --owner=user-2
neuro ps --name my-experiments-v1 -s failed -s succeeded
neuro ps --description=my favourite job
neuro ps -s failed -s succeeded -q

Options:

Name Description
-s, --status [pending | running | succeeded | failed | all] Filter out jobs by status (multiple option). Note: option all is deprecated, use neuro ps -a instead.
-o, --owner TEXT Filter out jobs by owner (multiple option).
-a, --all Show all jobs regardless the status (equivalent to \-s pending -s running -s succeeded -s failed)
-n, --name NAME Filter out jobs by name
-d, --description DESCRIPTION Filter out jobs by description (exact match)
-q, --quiet Run command in quiet mode (DEPRECATED)
-w, --wide Do not cut long lines for terminal width
--help Show this message and exit.

neuro job status

Display status of a job.

Usage:

neuro job status [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro job exec

Execute command in a running job.

Usage:

neuro job exec [OPTIONS] JOB CMD...

Examples:

# Provides a shell to the container:
neuro exec my-job /bin/bash

# Executes a single command in the container and returns the control:
neuro exec --no-tty my-job ls -l

Options:

Name Description
-t, --tty / -T, --no-tty Allocate virtual tty. Useful for interactive jobs.
--no-key-check Disable host key checks. Should be used with caution.
--timeout FLOAT Maximum allowed time for executing the command, 0 for no timeout [default: 0]
--help Show this message and exit.

neuro job port-forward

Forward port(s) of a running job to local port(s).

Usage:

neuro job port-forward [OPTIONS] JOB LOCAL_REMOTE_PORT...

Examples:

# Forward local port 2080 to port 80 of job's container.
# You can use http://localhost:2080 in browser to access job's served http
neuro job port-forward my-fastai-job 2080:80

# Forward local port 2222 to job's port 22
# Then copy all data from container's folder '/data' to current folder
# (please run second command in other terminal)
neuro job port-forward my-job-with-ssh-server 2222:22
rsync -avxzhe ssh -p 2222 root@localhost:/data .

# Forward few ports at once
neuro job port-forward my-job- 2080:80 2222:22 2000:100

Options:

Name Description
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro job logs

Print the logs for a container.

Usage:

neuro job logs [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro job kill

Kill job(s).

Usage:

neuro job kill [OPTIONS] JOBS...

Options:

Name Description
--help Show this message and exit.

neuro job top

Display GPU/CPU/Memory usage.

Usage:

neuro job top [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro job save

Save job's state to an image

Usage:

neuro job save [OPTIONS] JOB IMAGE

Examples:

neuro job save job-id image:ubuntu-patched
neuro job save my-favourite-job image://~/ubuntu-patched:v1
neuro job save my-favourite-job image://bob/ubuntu-patched

Options:

Name Description
--help Show this message and exit.

neuro job browse

Opens a job's URL in a web browser.

Usage:

neuro job browse [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro storage

Storage operations.

Usage:

neuro storage [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro storage cp Copy files and directories
neuro storage ls List directory contents
neuro storage glob List resources that match PATTERNS
neuro storage rm Remove files or directories
neuro storage mkdir Make directories
neuro storage mv Move or rename files and directories
neuro storage load Copy files and directories using MinIO (EXPERIMENTAL)

neuro storage cp

Copy files and directories.

Either SOURCES or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Use /dev/stdin and /dev/stdout file names to copy a file from terminal and
print the content of file on the storage to console.

Usage:

neuro storage cp [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# copy local files into remote storage root
neuro cp foo.txt bar/baz.dat storage:
neuro cp foo.txt bar/baz.dat -t storage:

# copy local directory `foo` into existing remote directory `bar`
neuro cp -r foo -t storage:bar

# copy the content of local directory `foo` into existing remote
# directory `bar`
neuro cp -r -T storage:foo storage:bar

# download remote file `foo.txt` into local file `/tmp/foo.txt` with
# explicit file:// scheme set
neuro cp storage:foo.txt file:///tmp/foo.txt
neuro cp -T storage:foo.txt file:///tmp/foo.txt
neuro cp storage:foo.txt file:///tmp
neuro cp storage:foo.txt -t file:///tmp

# download other user's remote file into the current directory
neuro cp storage://{username}/foo.txt .

# download only files with extension `.out` into the current directory
neuro cp storage:results/*.out .

Options:

Name Description
-r, --recursive Recursive copy, off by default
--glob / --no-glob Expand glob patterns in SOURCES with explicit scheme [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
-u, --update Copy only when the SOURCE file is newer than the destination file or when the destination file is missing
-p, --progress / -P, --no-progress Show progress, on by default
--help Show this message and exit.

neuro storage ls

List directory contents.

By default PATH is equal user's home dir (storage:)

Usage:

neuro storage ls [OPTIONS] [PATHS]...

Options:

Name Description
-h, --human-readable with -l print human readable sizes (e.g., 2K, 540M)
-l use a long listing format
--sort [name | size | time] sort by given field, default is name
-d, --directory list directories themselves, not their contents
--help Show this message and exit.

neuro storage glob

List resources that match PATTERNS.

Usage:

neuro storage glob [OPTIONS] [PATTERNS]...

Options:

Name Description
--help Show this message and exit.

neuro storage rm

Remove files or directories.

Usage:

neuro storage rm [OPTIONS] PATHS...

Examples:

neuro rm storage:foo/bar
neuro rm storage://{username}/foo/bar
neuro rm --recursive storage://{username}/foo/
neuro rm storage:foo/**/*.tmp

Options:

Name Description
-r, --recursive remove directories and their contents recursively
--glob / --no-glob Expand glob patterns in PATHS [default: True]
--help Show this message and exit.

neuro storage mkdir

Make directories.

Usage:

neuro storage mkdir [OPTIONS] PATHS...

Options:

Name Description
-p, --parents No error if existing, make parent directories as needed
--help Show this message and exit.

neuro storage mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro storage mv [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# move and rename remote file
neuro mv storage:foo.txt storage:bar/baz.dat
neuro mv -T storage:foo.txt storage:bar/baz.dat

# move remote files into existing remote directory
neuro mv storage:foo.txt storage:bar/baz.dat storage:dst
neuro mv storage:foo.txt storage:bar/baz.dat -t storage:dst

# move the content of remote directory into other existing
# remote directory
neuro mv -T storage:foo storage:bar

# move remote file into other user's directory
neuro mv storage:foo.txt storage://{username}/bar.dat

# move remote file from other user's directory
neuro mv storage://{username}/foo.txt storage:bar.dat

Options:

Name Description
--glob / --no-glob Expand glob patterns in SOURCES [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
--help Show this message and exit.

neuro storage load

Copy files and directories using MinIO (EXPERIMENTAL).

Same as "cp", but uses MinIO and the Amazon S3 protocol.

Usage:

neuro storage load [OPTIONS] [SOURCES]... [DESTINATION]

Options:

Name Description
-r, --recursive Recursive copy, off by default
--glob / --no-glob Expand glob patterns in SOURCES with explicit scheme [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
-u, --update Copy only when the SOURCE file is newer than the destination file or when the destination file is missing
-p, --progress Show progress, off by default
--help Show this message and exit.

neuro image

Container image operations.

Usage:

neuro image [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro image ls List images
neuro image push Push an image to platform registry
neuro image pull Pull an image from platform registry
neuro image tags List tags for image in platform registry

neuro image ls

List images.

Usage:

neuro image ls [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro image push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contain tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro image push [OPTIONS] LOCAL_IMAGE [REMOTE_IMAGE]

Examples:

neuro push myimage
neuro push alpine:latest image:my-alpine:production
neuro push alpine image://myfriend/alpine:shared

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro image pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro image pull [OPTIONS] REMOTE_IMAGE [LOCAL_IMAGE]

Examples:

neuro pull image:myimage
neuro pull image://myfriend/alpine:shared
neuro pull image://username/my-alpine:production alpine:from-registry

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro image tags

List tags for image in platform registry.

Image name must be URL with image:// scheme.

Usage:

neuro image tags [OPTIONS] IMAGE

Examples:

neuro image tags image://myfriend/alpine
neuro image tags image:myimage

Options:

Name Description
--help Show this message and exit.

neuro config

Client configuration.

Usage:

neuro config [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro config login Log into Neuromation Platform
neuro config login-with-token Log into Neuromation Platform with token
neuro config login-headless Log into Neuromation Platform from non-GUI server environment
neuro config show Print current settings
neuro config show-token Print current authorization token
neuro config docker Configure docker client for working with platform registry
neuro config logout Log out

neuro config login

Log into Neuromation Platform.

URL is a platform entrypoint URL.

Usage:

neuro config login [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro config login-with-token

Log into Neuromation Platform with token.

TOKEN is authentication token provided by Neuromation administration team.
URL is a platform entrypoint URL.

Usage:

neuro config login-with-token [OPTIONS] TOKEN [URL]

Options:

Name Description
--help Show this message and exit.

neuro config login-headless

Log into Neuromation Platform from non-GUI server environment.

URL is a platform entrypoint URL.

The command works similar to "neuro login" but instead of opening a browser
for performing OAuth registration prints an URL that should be open on guest
host.

Then user inputs a code displayed in a browser after successful login back
in neuro command to finish the login process.

Usage:

neuro config login-headless [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro config show

Print current settings.

Usage:

neuro config show [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config show-token

Print current authorization token.

Usage:

neuro config show-token [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config docker

Configure docker client for working with platform registry

Usage:

neuro config docker [OPTIONS]

Options:

Name Description
--docker-config PATH Specifies the location of the Docker client configuration files
--help Show this message and exit.

neuro config logout

Log out.

Usage:

neuro config logout [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro completion

Output shell completion code.

Usage:

neuro completion [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro completion generate Provide an instruction for shell completion generation
neuro completion patch Automatically patch shell configuration profile to enable completion

neuro completion generate

Provide an instruction for shell completion generation.

Usage:

neuro completion generate [OPTIONS] [bash|zsh]

Options:

Name Description
--help Show this message and exit.

neuro completion patch

Automatically patch shell configuration profile to enable completion

Usage:

neuro completion patch [OPTIONS] [bash|zsh]

Options:

Name Description
--help Show this message and exit.

neuro acl

Access Control List management.

Usage:

neuro acl [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro acl grant Shares resource specified by URI to a USER with PERMISSION Examples: neuro acl...
neuro acl revoke Revoke from a USER permissions for previously shared resource specified by URI...
neuro acl list List resource available to a USER or shared by a USER Examples: neuro acl list...

neuro acl grant

Shares resource specified by URI to a USER with PERMISSION

Usage:

neuro acl grant [OPTIONS] URI USER [read|write|manage]

Examples:

neuro acl grant storage:///sample_data/ alice manage
neuro acl grant image:resnet50 bob read
neuro acl grant job:///my_job_id alice write

Options:

Name Description
--help Show this message and exit.

neuro acl revoke

Revoke from a USER permissions for previously shared resource specified by
URI

Usage:

neuro acl revoke [OPTIONS] URI USER

Examples:

neuro acl revoke storage:///sample_data/ alice
neuro acl revoke image:resnet50 bob
neuro acl revoke job:///my_job_id alice

Options:

Name Description
--help Show this message and exit.

neuro acl list

List resource available to a USER or shared by a USER

Usage:

neuro acl list [OPTIONS]

Examples:

neuro acl list
neuro acl list --scheme storage
neuro acl list --shared
neuro acl list --shared --scheme image

Options:

Name Description
-s, --scheme TEXT Filter resources by scheme
--shared Output the resources shared by the user
--help Show this message and exit.

neuro help

Get help on a command.

Usage:

neuro help [OPTIONS] [COMMAND]...

Options:

Name Description
--help Show this message and exit.

neuro run

Run a job with predefined resources configuration.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro run [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest on a machine with smaller GPU resources
# (see exact values in `neuro config show`) and with two volumes mounted:
#   storage://<home-directory>   --> /var/storage/home (in read-write mode),
#   storage://neuromation/public --> /var/storage/neuromation (in read-only mode).
neuro run --preset=gpu-small --volume=HOME pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 and arg2 as its arguments:
neuro run -s cpu-small image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2

Options:

Name Description
-s, --preset PRESET Predefined resource configuration (to see available values, run neuro config show)
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container [default: 80]
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-n, --name NAME Optional job name
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage://~:/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
--help Show this message and exit.

neuro submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 arg2 arg3 as its arguments:
neuro submit image://~/my-ubuntu:latest --entrypoint=/script.sh arg1 arg2 arg3

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
--tpu-type TYPE TPU type to use
--tpu-sw-version VERSION Requested TPU software version
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-p, --preemptible / -P, --non-preemptible Run job on a lower-cost preemptible instance [default: False]
-n, --name NAME Optional job name
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage://~:/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
--help Show this message and exit.

neuro ps

List all jobs.

Usage:

neuro ps [OPTIONS]

Examples:

neuro ps -a
neuro ps -a --owner=user-1 --owner=user-2
neuro ps --name my-experiments-v1 -s failed -s succeeded
neuro ps --description=my favourite job
neuro ps -s failed -s succeeded -q

Options:

Name Description
-s, --status [pending | running | succeeded | failed | all] Filter out jobs by status (multiple option). Note: option all is deprecated, use neuro ps -a instead.
-o, --owner TEXT Filter out jobs by owner (multiple option).
-a, --all Show all jobs regardless the status (equivalent to \-s pending -s running -s succeeded -s failed)
-n, --name NAME Filter out jobs by name
-d, --description DESCRIPTION Filter out jobs by description (exact match)
-q, --quiet Run command in quiet mode (DEPRECATED)
-w, --wide Do not cut long lines for terminal width
--help Show this message and exit.

neuro status

Display status of a job.

Usage:

neuro status [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro exec

Execute command in a running job.

Usage:

neuro exec [OPTIONS] JOB CMD...

Examples:

# Provides a shell to the container:
neuro exec my-job /bin/bash

# Executes a single command in the container and returns the control:
neuro exec --no-tty my-job ls -l

Options:

Name Description
-t, --tty / -T, --no-tty Allocate virtual tty. Useful for interactive jobs.
--no-key-check Disable host key checks. Should be used with caution.
--timeout FLOAT Maximum allowed time for executing the command, 0 for no timeout [default: 0]
--help Show this message and exit.

neuro port-forward

Forward port(s) of a running job to local port(s).

Usage:

neuro port-forward [OPTIONS] JOB LOCAL_REMOTE_PORT...

Examples:

# Forward local port 2080 to port 80 of job's container.
# You can use http://localhost:2080 in browser to access job's served http
neuro job port-forward my-fastai-job 2080:80

# Forward local port 2222 to job's port 22
# Then copy all data from container's folder '/data' to current folder
# (please run second command in other terminal)
neuro job port-forward my-job-with-ssh-server 2222:22
rsync -avxzhe ssh -p 2222 root@localhost:/data .

# Forward few ports at once
neuro job port-forward my-job- 2080:80 2222:22 2000:100

Options:

Name Description
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro logs

Print the logs for a container.

Usage:

neuro logs [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro kill

Kill job(s).

Usage:

neuro kill [OPTIONS] JOBS...

Options:

Name Description
--help Show this message and exit.

neuro top

Display GPU/CPU/Memory usage.

Usage:

neuro top [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro save

Save job's state to an image

Usage:

neuro save [OPTIONS] JOB IMAGE

Examples:

neuro job save job-id image:ubuntu-patched
neuro job save my-favourite-job image://~/ubuntu-patched:v1
neuro job save my-favourite-job image://bob/ubuntu-patched

Options:

Name Description
--help Show this message and exit.

neuro login

Log into Neuromation Platform.

URL is a platform entrypoint URL.

Usage:

neuro login [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro logout

Log out.

Usage:

neuro logout [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro cp

Copy files and directories.

Either SOURCES or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Use /dev/stdin and /dev/stdout file names to copy a file from terminal and
print the content of file on the storage to console.

Usage:

neuro cp [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# copy local files into remote storage root
neuro cp foo.txt bar/baz.dat storage:
neuro cp foo.txt bar/baz.dat -t storage:

# copy local directory `foo` into existing remote directory `bar`
neuro cp -r foo -t storage:bar

# copy the content of local directory `foo` into existing remote
# directory `bar`
neuro cp -r -T storage:foo storage:bar

# download remote file `foo.txt` into local file `/tmp/foo.txt` with
# explicit file:// scheme set
neuro cp storage:foo.txt file:///tmp/foo.txt
neuro cp -T storage:foo.txt file:///tmp/foo.txt
neuro cp storage:foo.txt file:///tmp
neuro cp storage:foo.txt -t file:///tmp

# download other user's remote file into the current directory
neuro cp storage://{username}/foo.txt .

# download only files with extension `.out` into the current directory
neuro cp storage:results/*.out .

Options:

Name Description
-r, --recursive Recursive copy, off by default
--glob / --no-glob Expand glob patterns in SOURCES with explicit scheme [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
-u, --update Copy only when the SOURCE file is newer than the destination file or when the destination file is missing
-p, --progress / -P, --no-progress Show progress, on by default
--help Show this message and exit.

neuro ls

List directory contents.

By default PATH is equal user's home dir (storage:)

Usage:

neuro ls [OPTIONS] [PATHS]...

Options:

Name Description
-h, --human-readable with -l print human readable sizes (e.g., 2K, 540M)
-l use a long listing format
--sort [name | size | time] sort by given field, default is name
-d, --directory list directories themselves, not their contents
--help Show this message and exit.

neuro rm

Remove files or directories.

Usage:

neuro rm [OPTIONS] PATHS...

Examples:

neuro rm storage:foo/bar
neuro rm storage://{username}/foo/bar
neuro rm --recursive storage://{username}/foo/
neuro rm storage:foo/**/*.tmp

Options:

Name Description
-r, --recursive remove directories and their contents recursively
--glob / --no-glob Expand glob patterns in PATHS [default: True]
--help Show this message and exit.

neuro mkdir

Make directories.

Usage:

neuro mkdir [OPTIONS] PATHS...

Options:

Name Description
-p, --parents No error if existing, make parent directories as needed
--help Show this message and exit.

neuro mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro mv [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# move and rename remote file
neuro mv storage:foo.txt storage:bar/baz.dat
neuro mv -T storage:foo.txt storage:bar/baz.dat

# move remote files into existing remote directory
neuro mv storage:foo.txt storage:bar/baz.dat storage:dst
neuro mv storage:foo.txt storage:bar/baz.dat -t storage:dst

# move the content of remote directory into other existing
# remote directory
neuro mv -T storage:foo storage:bar

# move remote file into other user's directory
neuro mv storage:foo.txt storage://{username}/bar.dat

# move remote file from other user's directory
neuro mv storage://{username}/foo.txt storage:bar.dat

Options:

Name Description
--glob / --no-glob Expand glob patterns in SOURCES [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
--help Show this message and exit.

neuro images

List images.

Usage:

neuro images [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contain tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro push [OPTIONS] LOCAL_IMAGE [REMOTE_IMAGE]

Examples:

neuro push myimage
neuro push alpine:latest image:my-alpine:production
neuro push alpine image://myfriend/alpine:shared

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro pull [OPTIONS] REMOTE_IMAGE [LOCAL_IMAGE]

Examples:

neuro pull image:myimage
neuro pull image://myfriend/alpine:shared
neuro pull image://username/my-alpine:production alpine:from-registry

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro share

Shares resource specified by URI to a USER with PERMISSION

Usage:

neuro share [OPTIONS] URI USER [read|write|manage]

Examples:

neuro acl grant storage:///sample_data/ alice manage
neuro acl grant image:resnet50 bob read
neuro acl grant job:///my_job_id alice write

Options:

Name Description
--help Show this message and exit.

Api

TODO

Contributing

git clone https://github.com/neuromation/platform-api-clients.git
cd platform-api-clients/python

Before you begin, it is recommended to have clean virtual environment installed:

python -m venv .env
source .env/bin/activate

Development flow:

  • Install dependencies: make init
  • Run tests: make test
  • Lint: make lint
  • Publish to pypi: make publish

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

neuromation-19.9.2-py3-none-any.whl (92.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page