Keep your local python scripts installed and in sync with a databricks notebook. Shortens the feedback loop to develop projects using a hybrid enviroment
Project description
DB-Rocket
Keep your local python scripts installed and in sync with a databricks notebook. Every change on your local machine is automatically applied to the notebook. Shortens the feedback loop to develop git based projects. Removes the need to setup a local development environment.
Installation
pip3 install databricks-rocket
Please sure your python interpreter is 3.7 or higher.
Setup
Make sure you have a token installed for databricks cli (offical documentation). First, create a token for yourself on Databricks. Then set it up locally by running:
databricks configure --token
Then, the databricks token must be exported in your environment.
export DATABRICKS_TOKEN="mydatabrickstoken"
If your project is not a pip package already you have to turn it into one. You can use dbrocket to do that.
dbrocket setup
Will create a setup.py for you.
Using db-rocket
dbrocket launch
The command returns the exact command you have to perform in your notebook next.
Example:
We are now building your Python repo as a library...
Done! in your notebook install the library by running:
%pip install --upgrade pip
%pip install /dbfs/temp/username/databricks_rocket-1.1.3-py3-none-any.whl --force-reinstall
Create a cell in a notebook and paste the content (example below).
Support
- Databricks: >=7
- Python: >=3.7
- Tested on Platform: Linux, MacOs. Windows will probably not work but contributions are welcomed!
Acknowledgments
- Thanks Leon Poli for the Logo :)
- Thanks Stephane Leonard for source-code and documentation improvements :)
- Thanks Malachi Soord for the CICD setup and README improvements
Contributions are welcomed!
Security
For security issues please contact security@getyourguide.com.
Legal
db-rocket is licensed under the Apache License, Version 2.0. See LICENSE for the full text.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for databricks_rocket-1.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f9649dc71c482fa8bd04e1e956bd0d9a339ab9d1afc502f1d7d095f4efc3d7e5 |
|
MD5 | 1639a676a642ca0d4260e372219ffca8 |
|
BLAKE2b-256 | 78973df0b9d81e0a83e593cb43b0eb7d5084872b12c1fc144fb68bf170033176 |