Extract SQLs from digdag log
Project description
digdaglog2sql
Install
pip install --user digdaglog2sql
or,
$ python -m venv .venv
$ source .venv/bin/activate
(.venv)$ pip install digdaglog2sql
Usage
$ digdaglog2sql --help
Usage: digdaglog2sql [OPTIONS]
Options:
--input FILENAME Option is mutually exclusive with session_id,
site.
--session-id INTEGER Session ID of workflow. Option is mutually
exclusive with input.
--site [us|jp|eu01|ap02|ap03] Option is mutually exclusive with input.
--output FILENAME [required]
--drop-cdp-db If true, drop cdp_audience_xxx DB name.
--help Show this message and exit.
You can use log file on local environment.
digdaglog2sql --input workflow-log.txt --output output.sql
Or, you can use Session ID of Treasure Workflow.
digdaglog2sql --session-id 12345 --site us --output output.sql
Ensure set TD_API_KEY
into environment variable.
Note
As of May 5 2022, if you want to use sqllineage for Trino and Hive logs from Treasure Data, recommend to install sqlparse and sqllineage as the following:
pip install git+https://github.com/chezou/sqlparse.git@trino#egg=sqlparse==0.4.3.dev0
pip install git+https://github.com/chezou/sqllineage.git@trino#egg=sqllineage==1.3.4
You have to ensure to have node environment for sqllineage installation from source.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
digdaglog2sql-0.0.3.tar.gz
(7.8 kB
view hashes)
Built Distribution
Close
Hashes for digdaglog2sql-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a78bee78e19e62dd7279e08f15f802521452bb9e5f6f4ee2b55d3ef3a556b718 |
|
MD5 | c7dc96491618a76c8124cf9f826e4834 |
|
BLAKE2b-256 | de761711c9989e7572775703ce0f08b0d3971a10911e3b3f7fab130e25222159 |