Y-Scramble: a package for y-randomization validation
Project description
Y-Scramble
Y-Scramble is a simple python package to perform y-randomization validation
of machine learning models. It can be used for classification and regression tasks
and accepts models following the scikit-learn
inteface, and the user may use all scorers available at scikit-learn
(accuracy
, recall
, precision
).
Installing
Y-Scramble can be installed from PyPI using the following command:
$ pip install y-scamble
Usage
from y_scramble import Scrambler
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier()
scrambler = Scrambler(model=model, iterations=1000)
scores, zscores, pvalues, significances = scrambler.validate(
X, y,
scoring="accuracy",
cross_val_score_aggregator="mean",
pvalue_threshold=0.01
)
The scramble
object returns the scores, z-scores, p-values and the significancy
information for the model trained (base_model
) using the default dataset and for different randomized versions as well (scrambled_models
). These results are stores in numpy arrays, where the position of index 0 represents the base_model
and the others the scrambled_models
.
The score of the base_model
is stored in scores[0]
, and i's p-values is stored in
pvalues[0]
. If this p-value is significant, the value of significances[0]
will be
True
, indicating that base_model
shows a significantly better result when comparing to the randomized models. Following the same logic, scores[1]
to scores[1000]
, for example, will store the score values for the randomized model 1
and 1000
, respectively.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for y_scramble-0.0.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bf3610450f4bb7b6497708ef5a3db2f2dba9585f4d654745cbdcb4ab508df981 |
|
MD5 | 96694a2ed5c2464f620d5dcb50138fc8 |
|
BLAKE2b-256 | da13caf3d416168bbbf701d5b295f4d2cb9d1cb4fde87373f2db806fccf8952d |