Skip to main content

Neuromation Platform API client

Project description

codecov

Table of Contents

Preface

Welcome to Neuromation API Python client. Package ship command line tool called neuro. With neuro you can:

neuro

▇ ◣
▇ ◥ ◣
◣ ◥ ▇
▇ ◣ ▇
▇ ◥ ◣ ▇
▇ ◥ ▇ Neuromation Platform
▇ ◣ ◥
◥ ◣ ▇ Deep network training,
◥ ▇ inference and datasets

Usage:

neuro [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
-v, --verbose
--show-traceback
-u, --url TEXT
-t, --token TEXT
--version Show the version and exit.
--help Show this message and exit.

Commands:

neuro completion

Generates code to enable shell-completion.

Usage:

neuro completion [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

neuro completion generate

Provide an instruction for shell completion generation.

Usage:

neuro completion generate [OPTIONS]

Options:

Name Description
_--shell [bash zsh]_
--help Show this message and exit.

neuro completion patch

Automatically patch shell configuration profile to enable completion

Usage:

neuro completion patch [OPTIONS]

Options:

Name Description
_--shell [bash zsh]_
--help Show this message and exit.

neuro config

Client configuration settings commands.

Usage:

neuro config [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

FILE is being used for accessing remote shell, remote debug.

Note: this is temporal and going to be replaced in future by JWT token.

neuro config auth

Updates authorization token.

Usage:

neuro config auth [OPTIONS] TOKEN

Options:

Name Description
--insecure Store token in plain file instead system secured keyring
--help Show this message and exit.

neuro config forget

Forget authorization token.

Usage:

neuro config forget [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config id_rsa

Updates path to id_rsa file with private key.

FILE is being used for accessing remote shell, remote debug.

Note: this is temporal and going to be replaced in future by JWT token.

Usage:

neuro config id_rsa [OPTIONS] FILE

Options:

Name Description
--help Show this message and exit.

neuro config show

Prints current settings.

Usage:

neuro config show [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro config url

Updates settings with provided platform URL.

Usage:

neuro config url [OPTIONS] URL

Examples:

neuro config url https://platform.neuromation.io/api/v1

Options:

Name Description
--help Show this message and exit.

neuro help

Get help on a command

Usage:

neuro help [OPTIONS] [COMMAND]...

Options:

Name Description
--help Show this message and exit.

neuro image

Docker image operations

Usage:

neuro image [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

You will see here own and shared with you images

Remote image name must be URL with image:// scheme. Image names can contain tag.

Remote image must be URL with image:// scheme. Image names can contains tag. If tags not specified 'latest' will be used as value.

neuro image ls

List user's images which are available for jobs.

You will see here own and shared with you images

Usage:

neuro image ls [OPTIONS]

Options:

Name Description
--help Show this message and exit.

neuro image pull

Pull an image from platform registry.

Remote image name must be URL with image:// scheme. Image names can contain
tag.

Usage:

neuro image pull [OPTIONS] IMAGE_NAME [LOCAL_IMAGE_NAME]

Examples:

neuro image pull image:myimage
neuro image pull image://myfriend/alpine:shared
neuro image pull image://username/my-alpine:production alpine:from-registry

Options:

Name Description
--help Show this message and exit.

neuro image push

Push an image to platform registry.

Remote image must be URL with image:// scheme. Image names can contains tag.
If tags not specified 'latest' will be used as value.

Usage:

neuro image push [OPTIONS] IMAGE_NAME [REMOTE_IMAGE_NAME]

Examples:

neuro image push myimage
neuro image push alpine:latest image:my-alpine:production
neuro image push alpine image://myfriend/alpine:shared

Options:

Name Description
--help Show this message and exit.

neuro job

Job operations.

Usage:

neuro job [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

COMMANDS list will be passed as commands to model container.

neuro job exec

Executes command in a running job.

Usage:

neuro job exec [OPTIONS] ID CMD...

Options:

Name Description
-t, --tty Allocate virtual tty. Useful for interactive jobs.
--no-key-check Disable host key checks. Should be used with caution.
--help Show this message and exit.

neuro job kill

Kill job(s)

Usage:

neuro job kill [OPTIONS] ID...

Options:

Name Description
--help Show this message and exit.

neuro job list

List all jobs.

Usage:

neuro job list [OPTIONS]

Examples:

neuro job list --description="my favourite job"
neuro job list --status=all
neuro job list -s pending -s running -q

Options:

Name Description
_-s, --status [pending running
-d, --description DESCRIPTION Filter out job by job description (exact match)
-q, --quiet
--help Show this message and exit.

neuro job monitor

Monitor job output stream

Usage:

neuro job monitor [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro job ssh

Starts ssh terminal connected to running job. Job should be started with SSH
support enabled.

Usage:

neuro job ssh [OPTIONS] ID

Examples:

neuro job ssh --user alfa --key ./my_docker_id_rsa job-abc-def-ghk

Options:

Name Description
--user TEXT Container user name [default: root]
--key TEXT Path to container private key.
--help Show this message and exit.

neuro job status

Display status of a job

Usage:

neuro job status [OPTIONS] ID

Options:

Name Description
--help Show this message and exit.

neuro job submit

Start job using IMAGE.

COMMANDS list will be passed as commands to model container.

Usage:

neuro job submit [OPTIONS] IMAGE [CMD]...

Examples:

# Starts a container pytorch:latest with two paths mounted. Directory /q1/
# is mounted in read only mode to /qm directory within container.
# Directory /mod mounted to /mod directory in read-write mode.
neuro job submit --volume storage:/q1:/qm:ro --volume storage:/mod:/mod:rw     pytorch:latest

# Starts a container pytorch:latest with connection enabled to port 22 and
# sets PYTHONPATH environment value to /python.
# Please note that SSH server should be provided by container.
neuro job submit --env PYTHONPATH=/python --volume     storage:/data/2018q1:/data:ro --ssh 22 pytorch:latest

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm Request extended '/dev/shm' space
--http INTEGER Enable HTTP port forwarding to container
--ssh INTEGER Enable SSH port forwarding to container
--preemptible / --non-preemptible Run job on a lower-cost preemptible instance
-d, --description DESC Add optional description to the job
-q, --quiet Run command in quiet mode (print only job id)
--volume MOUNT Mounts directory from vault into container. Use multiple options to mount more than one volume
-e, --env VAR=VAL Set environment variable in container Use multiple options to define more than one variable
--env-file PATH File with environment variables to pass
--help Show this message and exit.

neuro model

Model operations.

Usage:

neuro model [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Job should be started with SSH support enabled.

The training job is created from IMAGE, dataset from DATASET and store output weights in RESULTS.

COMMANDS list will be passed as commands to model container.

neuro model debug

Starts ssh terminal connected to running job.

Job should be started with SSH support enabled.

Usage:

neuro model debug [OPTIONS] ID

Examples:

neuro model debug --localport 12789 job-abc-def-ghk

Options:

Name Description
--localport INTEGER Local port number for debug [default: 31234]
--help Show this message and exit.

neuro model train

Start training job using model.

The training job is created from IMAGE, dataset from DATASET and store
output weights in RESULTS.

COMMANDS list will be passed as commands to model container.

Usage:

neuro model train [OPTIONS] IMAGE DATASET RESULTS [CMD]...

Options:

Name Description
-g, --gpu NUMBER Number of GPUs to request [default: 0]
--gpu-model MODEL GPU to use [default: nvidia-tesla-k80]
-c, --cpu NUMBER Number of CPUs to request [default: 0.1]
-m, --memory AMOUNT Memory amount to request [default: 1G]
-x, --extshm Request extended '/dev/shm' space
--http INTEGER Enable HTTP port forwarding to container
--ssh INTEGER Enable SSH port forwarding to container
--preemptible / --non-preemptible Run job on a lower-cost preemptible instance
-d, --description DESC Add optional description to the job
-q, --quiet Run command in quiet mode (print only job id)
--help Show this message and exit.

neuro share

Shares resource specified by URI to a USER with PERMISSION

Usage:

neuro share [OPTIONS] URI USER [read|write|manage]

Examples:

neuro share storage:///sample_data/ alice manage neuro share
 image:resnet50 bob read neuro share job:///my_job_id alice write

Options:

Name Description
--help Show this message and exit.

neuro storage

Storage operations.

Usage:

neuro storage [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Either SOURCE or DESTINATION should have storage:// scheme. If scheme is omitted, file:// scheme is assumed.

By default PATH is equal user`s home dir (storage:)

SOURCE must contain path to the file or directory existing on the storage, and DESTINATION must contain the full path to the target file or directory.

neuro storage cp

Copy files and directories.

Either SOURCE or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Usage:

neuro storage cp [OPTIONS] SOURCE DESTINATION

Examples:

# copy local file ./foo into remote storage root
neuro storage cp ./foo storage:///
neuro storage cp ./foo storage:/

# download remote file foo into local file foo with
# explicit file:// scheme set
neuro storage cp storage:///foo file:///foo

Options:

Name Description
-r, --recursive Recursive copy, off by default
-p, --progress Show progress, off by default
--help Show this message and exit.

neuro storage ls

List directory contents.

By default PATH is equal user`s home dir (storage:)

Usage:

neuro storage ls [OPTIONS] [PATH]

Options:

Name Description
--help Show this message and exit.

neuro storage mkdir

Make directories.

Usage:

neuro storage mkdir [OPTIONS] PATH

Options:

Name Description
--help Show this message and exit.

neuro storage mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro storage mv [OPTIONS] SOURCE DESTINATION

Examples:

# move or rename remote file
neuro storage mv storage://{username}/foo.txt storage://{username}/bar.txt
neuro storage mv storage://{username}/foo.txt storage://~/bar/baz/foo.txt

# move or rename remote directory
neuro storage mv storage://{username}/foo/ storage://{username}/bar/
neuro storage mv storage://{username}/foo/ storage://{username}/bar/baz/foo/

Options:

Name Description
--help Show this message and exit.

neuro storage rm

Remove files or directories.

Usage:

neuro storage rm [OPTIONS] PATH

Examples:

neuro storage rm storage:///foo/bar/
neuro storage rm storage:/foo/bar/
neuro storage rm storage://{username}/foo/bar/

Options:

Name Description
--help Show this message and exit.

neuro store

Alias for storage (DEPRECATED)

Usage:

neuro store [OPTIONS] COMMAND [ARGS]...

Options:

Name Description
--help Show this message and exit.

Commands:

Either SOURCE or DESTINATION should have storage:// scheme. If scheme is omitted, file:// scheme is assumed.

By default PATH is equal user`s home dir (storage:)

SOURCE must contain path to the file or directory existing on the storage, and DESTINATION must contain the full path to the target file or directory.

neuro store cp

Copy files and directories.

Either SOURCE or DESTINATION should have storage:// scheme. If scheme is
omitted, file:// scheme is assumed.

Usage:

neuro store cp [OPTIONS] SOURCE DESTINATION

Examples:

# copy local file ./foo into remote storage root
neuro storage cp ./foo storage:///
neuro storage cp ./foo storage:/

# download remote file foo into local file foo with
# explicit file:// scheme set
neuro storage cp storage:///foo file:///foo

Options:

Name Description
-r, --recursive Recursive copy, off by default
-p, --progress Show progress, off by default
--help Show this message and exit.

neuro store ls

List directory contents.

By default PATH is equal user`s home dir (storage:)

Usage:

neuro store ls [OPTIONS] [PATH]

Options:

Name Description
--help Show this message and exit.

neuro store mkdir

Make directories.

Usage:

neuro store mkdir [OPTIONS] PATH

Options:

Name Description
--help Show this message and exit.

neuro store mv

Move or rename files and directories.

SOURCE must contain path to the file or directory existing on the storage,
and DESTINATION must contain the full path to the target file or directory.

Usage:

neuro store mv [OPTIONS] SOURCE DESTINATION

Examples:

# move or rename remote file
neuro storage mv storage://{username}/foo.txt storage://{username}/bar.txt
neuro storage mv storage://{username}/foo.txt storage://~/bar/baz/foo.txt

# move or rename remote directory
neuro storage mv storage://{username}/foo/ storage://{username}/bar/
neuro storage mv storage://{username}/foo/ storage://{username}/bar/baz/foo/

Options:

Name Description
--help Show this message and exit.

neuro store rm

Remove files or directories.

Usage:

neuro store rm [OPTIONS] PATH

Examples:

neuro storage rm storage:///foo/bar/
neuro storage rm storage:/foo/bar/
neuro storage rm storage://{username}/foo/bar/

Options:

Name Description
--help Show this message and exit.

Api

TODO

Contributing

git clone https://github.com/neuromation/platform-api-clients.git
cd platform-api-clients/python

Before you begin, it is recommended to have clean virtual environment installed:

python -m venv .env
source .env/bin/activate

Development flow:

  • Install dependencies: make init
  • Run tests: make test
  • Lint: make lint
  • Publish to pypi: make publish

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

neuromation-0.2.0b0-py3-none-any.whl (47.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page