Y-Scramble: a package for y-randomization validation
Project description
Y-Scramble
Y-Scramble is a simple python package to perform y-randomization validation
of machine learning models. It can be used for classification and regression tasks
and accepts models following the scikit-learn
inteface, and the user may use all scorers available at scikit-learn
(accuracy
, recall
, precision
).
Installing
Y-Scramble can be installed from PyPI using the following command:
$ pip install y-scamble
Usage
from y_scramble import Scrambler
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier()
scrambler = Scrambler(model=model, iterations=1000)
scores, zscores, pvalues, significances = scrambler.validate(
X, y,
scoring="accuracy",
cross_val_score_aggregator="mean",
pvalue_threshold=0.01
)
The scramble
object returns the scores, z-scores, p-values and the significancy
information for the model trained (base_model
) using the default dataset and for different randomized versions as well (scrambled_models
). These results are stores in numpy arrays, where the position of index 0 represents the base_model
and the others the scrambled_models
.
The score of the base_model
is stored in scores[0]
, and i's p-values is stored in
pvalues[0]
. If this p-value is significant, the value of significances[0]
will be
True
, indicating that base_model
shows a significantly better result when comparing to the randomized models. Following the same logic, scores[1]
to scores[1000]
, for example, will store the score values for the randomized model 1
and 1000
, respectively.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for y_scramble-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 243bffdb0c30eefd5866db2aa4625e7fec096e399eed98024683ac19ba8f3583 |
|
MD5 | 575d24c01ed23ee6a4c8ac40e67bb2d0 |
|
BLAKE2b-256 | 267c0db5162e54c7e9d28ac0782aed78cbc82bd1d9b4d0fee999efdd0867ac25 |