A minimalistic Google Storage client
Project description
gs is a command line interface (CLI) and Python library that provides a set of essential commands for Google Cloud Storage. It is modeled after the AWS CLI’s aws s3 command. Its features are:
Python 3 compatibility
A minimalistic set of dependencies
A tiny footprint
Intuitive convention-driven configuration of API credentials without browser login prompts
Checksum validation to ensure end-to-end data integrity in uploads and downloads
Progress bars for long-running upload and download operations
Resumable uploads and downloads
Multithreaded directory sync and batch delete, capable of handling large numbers of objects
An attractive paging and table layout interface
A JSON object metadata output mode for feeding data to other utilities
Installation
pip install gs
Synopsis
- Usage:
gs [OPTIONS] COMMAND [ARGS]...
- Options:
- --version
Show the version and exit.
- --help
Show this message and exit.
- Commands:
gs configure
Set gs config options, including the API key.
gs ls
List buckets or objects in a bucket/prefix.
gs cp
Copy files to, from, or between buckets.
gs mv
Move files to, from, or between buckets.
gs mb
Create a new Google Storage bucket.
gs rb
Permanently delete an empty bucket.
gs rm
Delete objects (files) from buckets.
gs sync
Sync a directory of files with bucket/prefix.
gs api
Use httpie to perform a raw HTTP API request.
gs presign
Get a pre-signed URL for accessing an object.
Run gs configure to configure Google service account access credentials that will be used by the gs command. You can create a new service account key at https://console.cloud.google.com/iam-admin/serviceaccounts.
Credentials
Before making API calls, gs ingests API credentials in the following order of priority:
First, gs checks if a GOOGLE_APPLICATION_CREDENTIALS environment variable is set. If so, it attempts to load and use credentials from the service account credentials filename referenced by the variable.
If that varible is not set, gs attempts to load service account credentials previously configured with gs configure (stored in ~/.config/gs/config.json).
If that fails, gs attempts to load a service account API token from Google instance metadata.
If that fails, gs prints a warning and attempts to make API requests anonymously.
Using the Python library interface
from gs import GSClient
client = GSClient()
object_meta = client.get("b/my-bucket/o/my-object")
with client.get("b/my-bucket/o/my-object", params=dict(alt="media"), stream=True) as res:
object_bytes = res.raw.read()
presigned_url = client.get_presigned_url("my-bucket", "my-object", expires_at=time.time()+3600)Links
Bugs
Please report bugs, issues, feature requests, etc. on GitHub.
License
Licensed under the terms of the MIT License.
