Skip to main content

Neuro Platform API client

Project description

codecov

Table of Contents

Preface

Welcome to Neuromation API Python client. Package ship command line tool called neuro. With neuro you can:

neuro

Usage:

neuro [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
-v, --verbose Give more output. Option is additive, and can be used up to 2 times.
-q, --quiet Give less output. Option is additive, and can be used up to 2 times.
--neuromation-config PATH Path to config directory.
--show-traceback Show python traceback on error, useful for debugging the tool.
--color [yes | no | auto] Color mode.
--disable-pypi-version-check Don't periodically check PyPI to determine whether a new version of Neuro Platform CLI is available for download.
--network-timeout FLOAT Network read timeout, seconds.
--version Show the version and exit.
--trace Trace sent HTTP requests and received replies to stderr.
--hide-token / --no-hide-token Prevent user's token sent in HTTP headers from being printed out to stderr during HTTP tracing. Can be used only together with option '--trace'. On by default.
--skip-stats / --no-skip-stats Skip sending usage statistics to Neuro servers. Note: the statistics has no sensitive data, e.g. file, job, image, or user names, executed command lines, environment variables, etc.
--help Show this message and exit.

Command Groups:

Usage Description
neuro admin Cluster administration commands
neuro job Job operations
neuro project Project operations
neuro storage Storage operations
neuro image Container image operations
neuro config Client configuration
neuro completion Output shell completion code
neuro acl Access Control List management
neuro blob Blob storage operations

Commands:

Usage Description
neuro help Get help on a command
neuro run Run a job with predefined resources configuration
neuro submit Submit an image to run on the cluster
neuro ps List all jobs
neuro status Display status of a job
neuro exec Execute command in a running job
neuro port-forward Forward port(s) of a running job to local port(s)
neuro logs Print the logs for a container
neuro kill Kill job(s)
neuro top Display GPU/CPU/Memory usage
neuro save Save job's state to an image
neuro login Log into Neuro Platform
neuro logout Log out
neuro cp Copy files and directories
neuro ls List directory contents
neuro rm Remove files or directories
neuro mkdir Make directories
neuro mv Move or rename files and directories
neuro images List images
neuro push Push an image to platform registry
neuro pull Pull an image from platform registry
neuro share Shares resource with another user

neuro admin

Cluster administration commands.

Usage:

neuro admin [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro admin get-clusters Print the list of available clusters
neuro admin generate-cluster-config Create a cluster configuration file
neuro admin add-cluster Create a new cluster and start its provisioning
neuro admin get-cluster-users Print the list of all users in the cluster with their assigned role
neuro admin add-cluster-user Add user access to specified cluster
neuro admin remove-cluster-user Remove user access from the cluster
neuro admin set-user-quota Set user quota to given values
neuro admin add-user-quota Add given values to user quota

neuro admin get-clusters

Print the list of available clusters.

Usage:

neuro admin get-clusters [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro admin generate-cluster-config

Create a cluster configuration file.

Usage:

neuro admin generate-cluster-config [OPTIONS] [CONFIG]

Options:

Name Description
--type [aws | gcp]
--help Show this message and exit.

neuro admin add-cluster

Create a new cluster and start its provisioning.

Usage:

neuro admin add-cluster [OPTIONS] CLUSTER_NAME CONFIG

Options:

Name Description
--help Show this message and exit.

neuro admin get-cluster-users

Print the list of all users in the cluster with their assigned role.

Usage:

neuro admin get-cluster-users [OPTIONS] [CLUSTER_NAME]

Options:

Name Description
--help Show this message and exit.

neuro admin add-cluster-user

Add user access to specified cluster.

The command supports one of 3 user roles: admin, manager or user.

Usage:

neuro admin add-cluster-user [OPTIONS] CLUSTER_NAME USER_NAME [ROLE]

Options:

Name Description
--help Show this message and exit.

neuro admin remove-cluster-user

Remove user access from the cluster.

Usage:

neuro admin remove-cluster-user [OPTIONS] CLUSTER_NAME USER_NAME

Options:

Name Description
--help Show this message and exit.

neuro admin set-user-quota

Set user quota to given values

Usage:

neuro admin set-user-quota [OPTIONS] CLUSTER_NAME USER_NAME

Options:

Name Description
-g, --gpu AMOUNT GPU quota value in hours (h) or minutes (m).
-n, --non-gpu AMOUNT Non-GPU quota value in hours (h) or minutes (m).
--help Show this message and exit.

neuro admin add-user-quota

Add given values to user quota

Usage:

neuro admin add-user-quota [OPTIONS] CLUSTER_NAME USER_NAME

Options:

Name Description
-g, --gpu AMOUNT Additional GPU quota value in hours (h) or minutes (m).
-n, --non-gpu AMOUNT Additional non-GPU quota value in hours (h) or minutes (m).
--help Show this message and exit.

neuro job

Job operations.

Usage:

neuro job [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro job run Run a job with predefined resources configuration
neuro job submit Submit an image to run on the cluster
neuro job ls List all jobs
neuro job status Display status of a job
neuro job tags List all tags submitted by the user
neuro job exec Execute command in a running job
neuro job port-forward Forward port(s) of a running job to local port(s)
neuro job logs Print the logs for a container
neuro job kill Kill job(s)
neuro job top Display GPU/CPU/Memory usage
neuro job save Save job's state to an image
neuro job browse Opens a job's URL in a web browser

neuro job run

Run a job with predefined resources configuration.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro job run [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest on a machine with smaller GPU resources
# (see exact values in `neuro config show`) and with two volumes mounted:
#   storage://<home-directory>   --> /var/storage/home (in read-write mode),
#   storage://neuromation/public --> /var/storage/neuromation (in read-only mode).
neuro run --preset=gpu-small --volume=HOME pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 and arg2 as its arguments:
neuro run -s cpu-small image:my-ubuntu:latest --entrypoint=/script.sh arg1 arg2

Options:

Name Description
-s, --preset PRESET Predefined resource configuration (to see available values, run neuro config show)
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container [default: 80]
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-n, --name NAME Optional job name
--tag TAG Optional job tag, multiple values allowed
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage::/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--restart [never | on-failure | always] Restart policy to apply when a job exits [default: never]
--life-span TIMEDELTA Optional job run-time limit in the format '1d2h3m4s' (some parts may be missing). Set '0' to disable. Default value '1d' can be changed in the user config.
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
-t, --tty Allocate a TTY
--help Show this message and exit.

neuro job submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro job submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 arg2 arg3 as its arguments:
neuro submit image:my-ubuntu:latest --entrypoint=/script.sh arg1 arg2 arg3

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
--tpu-type TYPE TPU type to use
--tpu-sw-version VERSION Requested TPU software version
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-p, --preemptible / -P, --non-preemptible Run job on a lower-cost preemptible instance [default: False]
-n, --name NAME Optional job name
--tag TAG Optional job tag, multiple values allowed
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage::/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--restart [never | on-failure | always] Restart policy to apply when a job exits [default: never]
--life-span TIMEDELTA Optional job run-time limit in the format '1d2h3m4s' (some parts may be missing). Set '0' to disable. Default value '1d' can be changed in the user config.
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
-t, --tty Allocate a TTY
--help Show this message and exit.

neuro job ls

List all jobs.

Usage:

neuro job ls [OPTIONS]

Examples:

neuro ps -a
neuro ps -a --owner=user-1 --owner=user-2
neuro ps --name my-experiments-v1 -s failed -s succeeded
neuro ps --description=my favourite job
neuro ps -s failed -s succeeded -q
neuro ps -t tag1 -t tag2

Options:

Name Description
-s, --status [pending | running | succeeded | failed | all] Filter out jobs by status (multiple option). Note: option all is deprecated, use neuro ps -a instead.
-o, --owner TEXT Filter out jobs by owner (multiple option).
-n, --name NAME Filter out jobs by name.
-t, --tag TAG Filter out jobs by tag (multiple option)
-d, --description DESCRIPTION Filter out jobs by description (exact match).
--since DATE Show jobs created after a specific date (including).
--until DATE Show jobs created before a specific date (including).
-a, --all Show all jobs regardless the status (equivalent to \-s pending -s running -s succeeded -s failed).
-q, --quiet Run command in quiet mode (DEPRECATED)
-w, --wide Do not cut long lines for terminal width.
--format COLUMNS Output table format, see "neuro help ps-format" for more info about the format specification. The default can be changed using the job.ps-format configuration variable documented in "neuro help user-config"
--full-uri Output full image URI.
--help Show this message and exit.

neuro job status

Display status of a job.

Usage:

neuro job status [OPTIONS] JOB

Options:

Name Description
--full-uri Output full URI.
--help Show this message and exit.

neuro job tags

List all tags submitted by the user.

Usage:

neuro job tags [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro job exec

Execute command in a running job.

Usage:

neuro job exec [OPTIONS] JOB CMD...

Examples:

# Provides a shell to the container:
neuro exec my-job /bin/bash

# Executes a single command in the container and returns the control:
neuro exec --no-tty my-job ls -l

Options:

Name Description
-t, --tty / -T, --no-tty Allocate virtual tty. Useful for interactive jobs.
-i, --interactive / -I, --no-interactive Keep STDIN open even if not attached. On for tty by default, false otherwise.
--timeout FLOAT Maximum allowed time for executing the command, 0 for no timeout [default: 0]
--help Show this message and exit.

neuro job port-forward

Forward port(s) of a running job to local port(s).

Usage:

neuro job port-forward [OPTIONS] JOB LOCAL_REMOTE_PORT...

Examples:

# Forward local port 2080 to port 80 of job's container.
# You can use http://localhost:2080 in browser to access job's served http
neuro job port-forward my-fastai-job 2080:80

# Forward local port 2222 to job's port 22
# Then copy all data from container's folder '/data' to current folder
# (please run second command in other terminal)
neuro job port-forward my-job-with-ssh-server 2222:22
rsync -avxzhe ssh -p 2222 root@localhost:/data .

# Forward few ports at once
neuro job port-forward my-job- 2080:80 2222:22 2000:100

Options:

Name Description
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro job logs

Print the logs for a container.

Usage:

neuro job logs [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro job kill

Kill job(s).

Usage:

neuro job kill [OPTIONS] JOBS...

Options:

Name Description
--help Show this message and exit.

neuro job top

Display GPU/CPU/Memory usage.

Usage:

neuro job top [OPTIONS] JOB

Options:

Name Description
--timeout FLOAT Maximum allowed time for executing the command, 0 for no timeout [default: 0]
--help Show this message and exit.

neuro job save

Save job's state to an image.

Usage:

neuro job save [OPTIONS] JOB IMAGE

Examples:

neuro job save job-id image:ubuntu-patched
neuro job save my-favourite-job image:ubuntu-patched:v1
neuro job save my-favourite-job image://bob/ubuntu-patched

Options:

Name Description
--help Show this message and exit.

neuro job browse

Opens a job's URL in a web browser.

Usage:

neuro job browse [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro project

Project operations.

Usage:

neuro project [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro project init Initialize an empty project

neuro project init

Initialize an empty project.

Usage:

neuro project init [OPTIONS] [SLUG]

Examples:

# Initializes a scaffolding for the new project with the recommended project
# structure (see http://github.com/neuromation/cookiecutter-neuro-project)
neuro project init

# Initializes a scaffolding for the new project with the recommended project
# structure and sets default project folder name to "example"
neuro project init my-project-id

Options:

Name Description
--help Show this message and exit.

neuro storage

Storage operations.

Usage:

neuro storage [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro storage cp Copy files and directories
neuro storage ls List directory contents
neuro storage glob List resources that match PATTERNS
neuro storage rm Remove files or directories
neuro storage mkdir Make directories
neuro storage mv Move or rename files and directories
neuro storage tree List contents of directories in a tree-like format
neuro storage load Copy files and directories using MinIO (EXPERIMENTAL)

neuro storage cp

Copy files and directories.

Either SOURCES or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Use /dev/stdin and /dev/stdout file names to copy a file from terminal and
print the content of file on the storage to console.

Usage:

neuro storage cp [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# copy local files into remote storage root
neuro cp foo.txt bar/baz.dat storage:
neuro cp foo.txt bar/baz.dat -t storage:

# copy local directory `foo` into existing remote directory `bar`
neuro cp -r foo -t storage:bar

# copy the content of local directory `foo` into existing remote
# directory `bar`
neuro cp -r -T storage:foo storage:bar

# download remote file `foo.txt` into local file `/tmp/foo.txt` with
# explicit file:// scheme set
neuro cp storage:foo.txt file:///tmp/foo.txt
neuro cp -T storage:foo.txt file:///tmp/foo.txt
neuro cp storage:foo.txt file:///tmp
neuro cp storage:foo.txt -t file:///tmp

# download other user's remote file into the current directory
neuro cp storage://{username}/foo.txt .

# download only files with extension `.out` into the current directory
neuro cp storage:results/*.out .

Options:

Name Description
-r, --recursive Recursive copy, off by default
--glob / --no-glob Expand glob patterns in SOURCES with explicit scheme. [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY.
-T, --no-target-directory Treat DESTINATION as a normal file.
-u, --update Copy only when the SOURCE file is newer than the destination file or when the destination file is missing.
--exclude Exclude files and directories that match the specified pattern. The default can be changed using the storage.cp-exclude configuration variable documented in "neuro help user-config"
--include Don't exclude files and directories that match the specified pattern. The default can be changed using the storage.cp-exclude configuration variable documented in "neuro help user-config"
-p, --progress / -P, --no-progress Show progress, on by default in TTY mode, off otherwise.
--help Show this message and exit.

neuro storage ls

List directory contents.

By default PATH is equal user's home dir (storage:)

Usage:

neuro storage ls [OPTIONS] [PATHS]...

Options:

Name Description
-a, --all do not ignore entries starting with .
-d, --directory list directories themselves, not their contents.
-h, --human-readable with -l print human readable sizes (e.g., 2K, 540M).
-l use a long listing format.
--sort [name | size | time] sort by given field, default is name.
--help Show this message and exit.

neuro storage glob

List resources that match PATTERNS.

Usage:

neuro storage glob [OPTIONS] [PATTERNS]...

Options:

Name Description
--help Show this message and exit.

neuro storage rm

Remove files or directories.

Usage:

neuro storage rm [OPTIONS] PATHS...

Examples:

neuro rm storage:foo/bar
neuro rm storage://{username}/foo/bar
neuro rm --recursive storage://{username}/foo/
neuro rm storage:foo/**/*.tmp

Options:

Name Description
-r, --recursive remove directories and their contents recursively
--glob / --no-glob Expand glob patterns in PATHS [default: True]
--help Show this message and exit.

neuro storage mkdir

Make directories.

Usage:

neuro storage mkdir [OPTIONS] PATHS...

Options:

Name Description
-p, --parents No error if existing, make parent directories as needed
--help Show this message and exit.

neuro storage mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro storage mv [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# move and rename remote file
neuro mv storage:foo.txt storage:bar/baz.dat
neuro mv -T storage:foo.txt storage:bar/baz.dat

# move remote files into existing remote directory
neuro mv storage:foo.txt storage:bar/baz.dat storage:dst
neuro mv storage:foo.txt storage:bar/baz.dat -t storage:dst

# move the content of remote directory into other existing
# remote directory
neuro mv -T storage:foo storage:bar

# move remote file into other user's directory
neuro mv storage:foo.txt storage://{username}/bar.dat

# move remote file from other user's directory
neuro mv storage://{username}/foo.txt storage:bar.dat

Options:

Name Description
--glob / --no-glob Expand glob patterns in SOURCES [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
--help Show this message and exit.

neuro storage tree

List contents of directories in a tree-like format.

Tree is a recursive directory listing program that produces a depth indented
listing of files, which is colorized ala dircolors if the LS_COLORS
environment variable is set and output is to tty. With no arguments, tree
lists the files in the storage: directory. When directory arguments are
given, tree lists all the files and/or directories found in the given
directories each in turn. Upon completion of listing all files/directories
found, tree returns the total number of files and/or directories listed.

By default PATH is equal user's home dir (storage:)

Usage:

neuro storage tree [OPTIONS] [PATH]

Options:

Name Description
-a, --all do not ignore entries starting with .
-h, --human-readable Print the size in a more human readable way.
-s, --size Print the size in bytes of each file.
--sort [name | size | time] sort by given field, default is name
--help Show this message and exit.

neuro storage load

Copy files and directories using MinIO (EXPERIMENTAL).

Same as "cp", but uses MinIO and the Amazon S3 protocol.
(DEPRECATED)

Usage:

neuro storage load [OPTIONS] [SOURCES]... [DESTINATION]

Options:

Name Description
-r, --recursive Recursive copy, off by default
--glob / --no-glob Expand glob patterns in SOURCES with explicit scheme [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
-u, --update Copy only when the SOURCE file is newer than the destination file or when the destination file is missing
-p, --progress Show progress, off by default
--help Show this message and exit.

neuro image

Container image operations.

Usage:

neuro image [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro image ls List images
neuro image push Push an image to platform registry
neuro image pull Pull an image from platform registry
neuro image tags List tags for image in platform registry

neuro image ls

List images.

Usage:

neuro image ls [OPTIONS]

Options:

Name Description
-l List in long format.
--full-uri Output full image URI.
--help Show this message and exit.

neuro image push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contain tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro image push [OPTIONS] LOCAL_IMAGE [REMOTE_IMAGE]

Examples:

neuro push myimage
neuro push alpine:latest image:my-alpine:production
neuro push alpine image://myfriend/alpine:shared

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro image pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro image pull [OPTIONS] REMOTE_IMAGE [LOCAL_IMAGE]

Examples:

neuro pull image:myimage
neuro pull image://myfriend/alpine:shared
neuro pull image://username/my-alpine:production alpine:from-registry

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro image tags

List tags for image in platform registry.

Image name must be URL with image:// scheme.

Usage:

neuro image tags [OPTIONS] IMAGE

Examples:

neuro image tags image://myfriend/alpine
neuro image tags image:myimage

Options:

Name Description
--help Show this message and exit.

neuro config

Client configuration.

Usage:

neuro config [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro config login Log into Neuro Platform
neuro config login-with-token Log into Neuro Platform with token
neuro config login-headless Log into Neuro Platform from non-GUI server environment
neuro config show Print current settings
neuro config show-token Print current authorization token
neuro config show-quota Print quota and remaining computation time for active cluster
neuro config aliases List available command aliases
neuro config get-clusters Fetch and display the list of available clusters
neuro config switch-cluster Switch the active cluster
neuro config docker Configure docker client to fit the Neuro Platform
neuro config logout Log out

neuro config login

Log into Neuro Platform.

URL is a platform entrypoint URL.

Usage:

neuro config login [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro config login-with-token

Log into Neuro Platform with token.

TOKEN is authentication token provided by administration team. URL is a
platform entrypoint URL.

Usage:

neuro config login-with-token [OPTIONS] TOKEN [URL]

Options:

Name Description
--help Show this message and exit.

neuro config login-headless

Log into Neuro Platform from non-GUI server environment.

URL is a platform entrypoint URL.

The command works similar to "neuro login" but instead of opening a browser
for performing OAuth registration prints an URL that should be open on guest
host.

Then user inputs a code displayed in a browser after successful login back
in neuro command to finish the login process.

Usage:

neuro config login-headless [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro config show

Print current settings.

Usage:

neuro config show [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config show-token

Print current authorization token.

Usage:

neuro config show-token [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config show-quota

Print quota and remaining computation time for active cluster.

Usage:

neuro config show-quota [OPTIONS] [USER]

Options:

Name Description
--help Show this message and exit.

neuro config aliases

List available command aliases.

Usage:

neuro config aliases [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config get-clusters

Fetch and display the list of available clusters.

Usage:

neuro config get-clusters [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config switch-cluster

Switch the active cluster.

CLUSTER_NAME is the cluster name to select. The interactive prompt is used
if the name is omitted (default).

Usage:

neuro config switch-cluster [OPTIONS] [CLUSTER_NAME]

Options:

Name Description
--help Show this message and exit.

neuro config docker

Configure docker client to fit the Neuro Platform.

Usage:

neuro config docker [OPTIONS]

Options:

Name Description
--docker-config PATH Specifies the location of the Docker client configuration files
--help Show this message and exit.

neuro config logout

Log out.

Usage:

neuro config logout [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro completion

Output shell completion code.

Usage:

neuro completion [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro completion generate Provide an instruction for shell completion generation
neuro completion patch Automatically patch shell configuration profile to enable completion

neuro completion generate

Provide an instruction for shell completion generation.

Usage:

neuro completion generate [OPTIONS] [bash|zsh]

Options:

Name Description
--help Show this message and exit.

neuro completion patch

Automatically patch shell configuration profile to enable completion

Usage:

neuro completion patch [OPTIONS] [bash|zsh]

Options:

Name Description
--help Show this message and exit.

neuro acl

Access Control List management.

Usage:

neuro acl [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro acl grant Shares resource with another user
neuro acl revoke Revoke user access from another user
neuro acl list List shared resources

neuro acl grant

Shares resource with another user.

URI shared resource.

USER username to share resource with.

PERMISSION sharing access right: read, write, or manage.

Usage:

neuro acl grant [OPTIONS] URI USER [read|write|manage]

Examples:

neuro acl grant storage:///sample_data/ alice manage
neuro acl grant image:resnet50 bob read
neuro acl grant job:///my_job_id alice write

Options:

Name Description
--help Show this message and exit.

neuro acl revoke

Revoke user access from another user.

URI previously shared resource to revoke.

USER to revoke URI resource from.

Usage:

neuro acl revoke [OPTIONS] URI USER

Examples:

neuro acl revoke storage:///sample_data/ alice
neuro acl revoke image:resnet50 bob
neuro acl revoke job:///my_job_id alice

Options:

Name Description
--help Show this message and exit.

neuro acl list

List shared resources.

The command displays a list of resources shared BY current user (default).

To display a list of resources shared WITH current user apply --shared
option.

Usage:

neuro acl list [OPTIONS]

Examples:

neuro acl list
neuro acl list --scheme storage
neuro acl list --shared
neuro acl list --shared --scheme image

Options:

Name Description
-u TEXT Use specified user or role.
-s, --scheme TEXT Filter resources by scheme, e.g. job, storage, image or user.
--shared Output the resources shared by the user.
--full-uri Output full URI.
--help Show this message and exit.

neuro blob

Blob storage operations.

Usage:

neuro blob [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Usage Description
neuro blob cp Simple utility to copy files and directories into and from Blob Storage
neuro blob ls List buckets or bucket contents
neuro blob glob List resources that match PATTERNS

neuro blob cp

Simple utility to copy files and directories into and from Blob Storage.
Either SOURCES or DESTINATION should have blob:// scheme. If scheme is
omitted, file:// scheme is assumed. It is currently not possible to copy
files between Blob Storage (blob://) destination, nor with storage://
scheme paths. Use /dev/stdin and /dev/stdout file names to upload a file
from standard input or output to stdout. File permissions, modification
times and other attributes will not be passed to Blob Storage metadata
during upload.

Usage:

neuro blob cp [OPTIONS] [SOURCES]... [DESTINATION]

Options:

Name Description
-r, --recursive Recursive copy, off by default
--glob / --no-glob Expand glob patterns in SOURCES with explicit scheme. [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY.
-T, --no-target-directory Treat DESTINATION as a normal file.
--exclude Exclude files and directories that match the specified pattern. The default can be changed using the storage.cp-exclude configuration variable documented in "neuro help user-config"
--include Don't exclude files and directories that match the specified pattern. The default can be changed using the storage.cp-exclude configuration variable documented in "neuro help user-config"
-p, --progress / -P, --no-progress Show progress, on by default.
--help Show this message and exit.

neuro blob ls

List buckets or bucket contents.

Usage:

neuro blob ls [OPTIONS] [PATHS]...

Options:

Name Description
-h, --human-readable with -l print human readable sizes (e.g., 2K, 540M).
-l use a long listing format.
--sort [name | size | time] sort by given field, default is name.
-r, --recursive List all keys under the URL path provided, not just 1 level depths.
--help Show this message and exit.

neuro blob glob

List resources that match PATTERNS.

Usage:

neuro blob glob [OPTIONS] [PATTERNS]...

Options:

Name Description
--help Show this message and exit.

neuro help

Get help on a command.

Usage:

neuro help [OPTIONS] [COMMAND]...

Options:

Name Description
--help Show this message and exit.

neuro run

Run a job with predefined resources configuration.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro run [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest on a machine with smaller GPU resources
# (see exact values in `neuro config show`) and with two volumes mounted:
#   storage://<home-directory>   --> /var/storage/home (in read-write mode),
#   storage://neuromation/public --> /var/storage/neuromation (in read-only mode).
neuro run --preset=gpu-small --volume=HOME pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 and arg2 as its arguments:
neuro run -s cpu-small image:my-ubuntu:latest --entrypoint=/script.sh arg1 arg2

Options:

Name Description
-s, --preset PRESET Predefined resource configuration (to see available values, run neuro config show)
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container [default: 80]
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-n, --name NAME Optional job name
--tag TAG Optional job tag, multiple values allowed
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage::/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--restart [never | on-failure | always] Restart policy to apply when a job exits [default: never]
--life-span TIMEDELTA Optional job run-time limit in the format '1d2h3m4s' (some parts may be missing). Set '0' to disable. Default value '1d' can be changed in the user config.
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
-t, --tty Allocate a TTY
--help Show this message and exit.

neuro submit

Submit an image to run on the cluster.

IMAGE container image name.

CMD list will be passed as commands to model container.

Usage:

neuro submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw pytorch:latest

# Starts a container using the custom image my-ubuntu:latest stored in neuromation
# registry, run /script.sh and pass arg1 arg2 arg3 as its arguments:
neuro submit image:my-ubuntu:latest --entrypoint=/script.sh arg1 arg2 arg3

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
--tpu-type TYPE TPU type to use
--tpu-sw-version VERSION Requested TPU software version
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm / -X, --no-extshm Request extended '/dev/shm' space [default: True]
--http PORT Enable HTTP port forwarding to container
--http-auth / --no-http-auth Enable HTTP authentication for forwarded HTTP port [default: True]
-p, --preemptible / -P, --non-preemptible Run job on a lower-cost preemptible instance [default: False]
-n, --name NAME Optional job name
--tag TAG Optional job tag, multiple values allowed
-d, --description DESC Optional job description in free format
-q, --quiet Run command in quiet mode (DEPRECATED)
-v, --volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume. --volume=HOME is an alias for storage::/var/storage/home:rw and storage://neuromation/public:/var/storage/neuromation:ro
--entrypoint TEXT Executable entrypoint in the container (note that it overwrites ENTRYPOINT and CMD instructions of the docker image)
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--restart [never | on-failure | always] Restart policy to apply when a job exits [default: never]
--life-span TIMEDELTA Optional job run-time limit in the format '1d2h3m4s' (some parts may be missing). Set '0' to disable. Default value '1d' can be changed in the user config.
--wait-start / --no-wait-start Wait for a job start or failure [default: True]
--pass-config / --no-pass-config Upload neuro config to the job [default: False]
--browse Open a job's URL in a web browser
--detach Don't attach to job logs and don't wait for exit code
-t, --tty Allocate a TTY
--help Show this message and exit.

neuro ps

List all jobs.

Usage:

neuro ps [OPTIONS]

Examples:

neuro ps -a
neuro ps -a --owner=user-1 --owner=user-2
neuro ps --name my-experiments-v1 -s failed -s succeeded
neuro ps --description=my favourite job
neuro ps -s failed -s succeeded -q
neuro ps -t tag1 -t tag2

Options:

Name Description
-s, --status [pending | running | succeeded | failed | all] Filter out jobs by status (multiple option). Note: option all is deprecated, use neuro ps -a instead.
-o, --owner TEXT Filter out jobs by owner (multiple option).
-n, --name NAME Filter out jobs by name.
-t, --tag TAG Filter out jobs by tag (multiple option)
-d, --description DESCRIPTION Filter out jobs by description (exact match).
--since DATE Show jobs created after a specific date (including).
--until DATE Show jobs created before a specific date (including).
-a, --all Show all jobs regardless the status (equivalent to \-s pending -s running -s succeeded -s failed).
-q, --quiet Run command in quiet mode (DEPRECATED)
-w, --wide Do not cut long lines for terminal width.
--format COLUMNS Output table format, see "neuro help ps-format" for more info about the format specification. The default can be changed using the job.ps-format configuration variable documented in "neuro help user-config"
--full-uri Output full image URI.
--help Show this message and exit.

neuro status

Display status of a job.

Usage:

neuro status [OPTIONS] JOB

Options:

Name Description
--full-uri Output full URI.
--help Show this message and exit.

neuro exec

Execute command in a running job.

Usage:

neuro exec [OPTIONS] JOB CMD...

Examples:

# Provides a shell to the container:
neuro exec my-job /bin/bash

# Executes a single command in the container and returns the control:
neuro exec --no-tty my-job ls -l

Options:

Name Description
-t, --tty / -T, --no-tty Allocate virtual tty. Useful for interactive jobs.
-i, --interactive / -I, --no-interactive Keep STDIN open even if not attached. On for tty by default, false otherwise.
--timeout FLOAT Maximum allowed time for executing the command, 0 for no timeout [default: 0]
--help Show this message and exit.

neuro port-forward

Forward port(s) of a running job to local port(s).

Usage:

neuro port-forward [OPTIONS] JOB LOCAL_REMOTE_PORT...

Examples:

# Forward local port 2080 to port 80 of job's container.
# You can use http://localhost:2080 in browser to access job's served http
neuro job port-forward my-fastai-job 2080:80

# Forward local port 2222 to job's port 22
# Then copy all data from container's folder '/data' to current folder
# (please run second command in other terminal)
neuro job port-forward my-job-with-ssh-server 2222:22
rsync -avxzhe ssh -p 2222 root@localhost:/data .

# Forward few ports at once
neuro job port-forward my-job- 2080:80 2222:22 2000:100

Options:

Name Description
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro logs

Print the logs for a container.

Usage:

neuro logs [OPTIONS] JOB

Options:

Name Description
--help Show this message and exit.

neuro kill

Kill job(s).

Usage:

neuro kill [OPTIONS] JOBS...

Options:

Name Description
--help Show this message and exit.

neuro top

Display GPU/CPU/Memory usage.

Usage:

neuro top [OPTIONS] JOB

Options:

Name Description
--timeout FLOAT Maximum allowed time for executing the command, 0 for no timeout [default: 0]
--help Show this message and exit.

neuro save

Save job's state to an image.

Usage:

neuro save [OPTIONS] JOB IMAGE

Examples:

neuro job save job-id image:ubuntu-patched
neuro job save my-favourite-job image:ubuntu-patched:v1
neuro job save my-favourite-job image://bob/ubuntu-patched

Options:

Name Description
--help Show this message and exit.

neuro login

Log into Neuro Platform.

URL is a platform entrypoint URL.

Usage:

neuro login [OPTIONS] [URL]

Options:

Name Description
--help Show this message and exit.

neuro logout

Log out.

Usage:

neuro logout [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro cp

Copy files and directories.

Either SOURCES or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Use /dev/stdin and /dev/stdout file names to copy a file from terminal and
print the content of file on the storage to console.

Usage:

neuro cp [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# copy local files into remote storage root
neuro cp foo.txt bar/baz.dat storage:
neuro cp foo.txt bar/baz.dat -t storage:

# copy local directory `foo` into existing remote directory `bar`
neuro cp -r foo -t storage:bar

# copy the content of local directory `foo` into existing remote
# directory `bar`
neuro cp -r -T storage:foo storage:bar

# download remote file `foo.txt` into local file `/tmp/foo.txt` with
# explicit file:// scheme set
neuro cp storage:foo.txt file:///tmp/foo.txt
neuro cp -T storage:foo.txt file:///tmp/foo.txt
neuro cp storage:foo.txt file:///tmp
neuro cp storage:foo.txt -t file:///tmp

# download other user's remote file into the current directory
neuro cp storage://{username}/foo.txt .

# download only files with extension `.out` into the current directory
neuro cp storage:results/*.out .

Options:

Name Description
-r, --recursive Recursive copy, off by default
--glob / --no-glob Expand glob patterns in SOURCES with explicit scheme. [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY.
-T, --no-target-directory Treat DESTINATION as a normal file.
-u, --update Copy only when the SOURCE file is newer than the destination file or when the destination file is missing.
--exclude Exclude files and directories that match the specified pattern. The default can be changed using the storage.cp-exclude configuration variable documented in "neuro help user-config"
--include Don't exclude files and directories that match the specified pattern. The default can be changed using the storage.cp-exclude configuration variable documented in "neuro help user-config"
-p, --progress / -P, --no-progress Show progress, on by default in TTY mode, off otherwise.
--help Show this message and exit.

neuro ls

List directory contents.

By default PATH is equal user's home dir (storage:)

Usage:

neuro ls [OPTIONS] [PATHS]...

Options:

Name Description
-a, --all do not ignore entries starting with .
-d, --directory list directories themselves, not their contents.
-h, --human-readable with -l print human readable sizes (e.g., 2K, 540M).
-l use a long listing format.
--sort [name | size | time] sort by given field, default is name.
--help Show this message and exit.

neuro rm

Remove files or directories.

Usage:

neuro rm [OPTIONS] PATHS...

Examples:

neuro rm storage:foo/bar
neuro rm storage://{username}/foo/bar
neuro rm --recursive storage://{username}/foo/
neuro rm storage:foo/**/*.tmp

Options:

Name Description
-r, --recursive remove directories and their contents recursively
--glob / --no-glob Expand glob patterns in PATHS [default: True]
--help Show this message and exit.

neuro mkdir

Make directories.

Usage:

neuro mkdir [OPTIONS] PATHS...

Options:

Name Description
-p, --parents No error if existing, make parent directories as needed
--help Show this message and exit.

neuro mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro mv [OPTIONS] [SOURCES]... [DESTINATION]

Examples:

# move and rename remote file
neuro mv storage:foo.txt storage:bar/baz.dat
neuro mv -T storage:foo.txt storage:bar/baz.dat

# move remote files into existing remote directory
neuro mv storage:foo.txt storage:bar/baz.dat storage:dst
neuro mv storage:foo.txt storage:bar/baz.dat -t storage:dst

# move the content of remote directory into other existing
# remote directory
neuro mv -T storage:foo storage:bar

# move remote file into other user's directory
neuro mv storage:foo.txt storage://{username}/bar.dat

# move remote file from other user's directory
neuro mv storage://{username}/foo.txt storage:bar.dat

Options:

Name Description
--glob / --no-glob Expand glob patterns in SOURCES [default: True]
-t, --target-directory DIRECTORY Copy all SOURCES into DIRECTORY
-T, --no-target-directory Treat DESTINATION as a normal file
--help Show this message and exit.

neuro images

List images.

Usage:

neuro images [OPTIONS]

Options:

Name Description
-l List in long format.
--full-uri Output full image URI.
--help Show this message and exit.

neuro push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contain tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro push [OPTIONS] LOCAL_IMAGE [REMOTE_IMAGE]

Examples:

neuro push myimage
neuro push alpine:latest image:my-alpine:production
neuro push alpine image://myfriend/alpine:shared

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro pull [OPTIONS] REMOTE_IMAGE [LOCAL_IMAGE]

Examples:

neuro pull image:myimage
neuro pull image://myfriend/alpine:shared
neuro pull image://username/my-alpine:production alpine:from-registry

Options:

Name Description
-q, --quiet Run command in quiet mode (DEPRECATED)
--help Show this message and exit.

neuro share

Shares resource with another user.

URI shared resource.

USER username to share resource with.

PERMISSION sharing access right: read, write, or manage.

Usage:

neuro share [OPTIONS] URI USER [read|write|manage]

Examples:

neuro acl grant storage:///sample_data/ alice manage
neuro acl grant image:resnet50 bob read
neuro acl grant job:///my_job_id alice write

Options:

Name Description
--help Show this message and exit.

Api

TODO

Contributing

git clone https://github.com/neuromation/platform-api-clients.git
cd platform-api-clients/python

For OSX users install coreutils to properly interpret shell commands:

brew install coreutils

Before you begin, it is recommended to have clean virtual environment installed:

python -m venv .env
source .env/bin/activate

Development flow:

  • Install dependencies: make init
  • Run tests: make test
  • Lint: make lint
  • Publish to pypi: make publish

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

neuromation-20.6.10.tar.gz (279.6 kB view hashes)

Uploaded Source

Built Distribution

neuromation-20.6.10-py3-none-any.whl (154.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page