sqlite-ml

An SQLite extension for machine learning

Train machine learning models and run predictions directly from your SQLite database. Inspired by PostgresML.

Why?

Why bother running Machine Learning workloads in SQLite? Good question! Here are some answers:

• Machine Learning number one problem is data
• Instead of trying to bring the data to the model, why not bring a model along the data?
• Instead of exporting the data some place else, training a model, performing inference and bringing back the prediction into the database, why not do that directly alonside the data?
• Lots of ETL/ELT workloads are converging to pure SQL processing, why not do that also with predictions?
• The field of MLOps tries to unify the ML lifecycle but this is hard when working on multiple environments
• SQLite is fast
• The SQLite ecosystem is quite good (take a look at all the wonderful things around Datasette)
• SQLite is often used for ad-hoc data analysis, why not give the opportunity to also make predictions at the same time?
• Easy to integrate predictions for existings applications, use a simple SQL query!

Inspiration

All the things on the internet that has inspired this project:

• PostgresML
• SQLite Run-Time Loadable Extensions
• Alex Garcia's sqlite-loadable-rs
• Alex Garcia's SQLite extensions
• Alex Garcia, "Making SQLite extensions pip install-able"
• Ryan Patterson's sqlite3_ext
• Max Halford, "Online gradient descent written in SQL"
• Ricardo Anderegg, "Extending SQLite with Rust"
• Who needs MLflow when you have SQLite?
• PostgresML is Moving to Rust for our 2.0 Release
• The Virtual Table Mechanism Of SQLite
• PyO3
• NoiSQL — Generating Music With SQL Queries