Raise red flags on machine learning datasets.
Project description
Redflag
🚩 redflag
aims to be an automatic safety net for machine learning datasets. The vision is to accept input of a Pandas DataFrame
or NumPy ndarray
(one for each of the input X
and target y
in a machine learning task). redflag
will provide an analysis of each feature, and of the target, including aspects such as class imbalance, outliers, anomalous data patterns, threats to the IID assumption, and so on. The goal is to complement other projects like pandas-profiling
and greatexpectations
.
This project is very rough and does not do much yet. The API will very likely change without warning. Please consider contributing!
Installation
You can install this package with pip
:
pip install redflag
For developers, there are pip
options for installing tests
, docs
and dev
dependencies, e.g. pip install redflag[dev]
to install all testing and documentation packages.
Example
redflag
is currently just a collection of functions. Most of the useful ones take a single column of data (e.g. a 1D NumPy array) and run a single test. For example, we can do some outlier detection. The get_outliers()
function returns the indices of data points that are considered outliers:
>>> import redflag as rf
>>> data = [-3, -2, -2, -1, 0, 0, 0, 1, 2, 2, 3]
>>> rf.get_outliers(data)
array([], dtype=int64)
>>> rf.get_outliers(3 * data + [100])
array([33])
See the documentation, and specifically the notebook Basic_usage.ipynb for several other basic examples.
Documentation
Contributing
Please see CONTRIBUTING.md
. There is also a section in the documentation about Development.
Testing
You can run the tests (requires pytest
and pytest-cov
) with
pytest
Most of the tests are doctests, but pytest
will run them using the settings in setup.cfg
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for redflag-0.1.9rc2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eabddf78201329c94b12d0c0efb2b003156e9fa4c50dd9bbb3576f49dc4b2898 |
|
MD5 | 8e96f9b9f46fa9ee0753b2e5ace76bb5 |
|
BLAKE2b-256 | 4dc5fbc6e82e88f7ba6c2b5166b9d7b54734d43b1653f7ecc8f4085fb113a4cf |