laforge 0.1.6

laforge is a low-key build system for working with data.

💻 Introduction

You know, I've always thought technology could solve almost any problem.

$ python -m pip install laforge -q
...

$ laforge create
Creating /home/matvan/science/build.ini
? Creating a new laforge INI at:  /home/matvan/science/build.ini

Creating /home/matvan/science/build.ini

? Default read directory, relative to /home/matvan/science/:  ./data
? Default write directory, relative to /home/matvan/science/:  ./output
? Default execute directory, relative to /home/matvan/science/:  ./
? SQL Distribution:  SQLite
?     Database:  :memory:
New laforge INI written at: /home/matvan/science/build.ini
Enjoy!

There's theory and then there's application. They don't always jibe.

$ vim ./build.ini
...

$ cat ./build.ini
[DEFAULT]
read_dir: ./data
write_dir: ./output
execute_dir: ./
distro: sqlite
database: :memory:

# Write the contents of an Excel sheet as a SQL table
[load_individual]
read: 2019_indiv_data.xlsx
write: raw_grp

# Write the contents of a CSV as a SQL table
[load_group]
read: 2019_grp_data.csv
write: raw_indiv

# Execute a standalone SQL script; read a SQL table and save as CSV
[do_some_things]
execute: do_stuff.sql
read: laforge.demo.aggregate
write: aggregate.csv

# Read the result of an ad-hoc SQL query; write to an HTML table
[peek]
read:
    "select top 50 *
    from demo.aggregate agg
    left join demo.raw_indiv ri
        on agg.v1 = r1.v2
    order by newid();"
write: results_peek.html

Yeah, but that's imposs—yes, sir.

$ laforge build

‖-)

📓 Documentation

https://laforge.readthedocs.io/en/latest/

⚗️ Development

Captain, we can do it... It'll take fifteen years and a research team of a hundred.

🤖 Automation

🚀 Downstream

🧙‍ Author