DESlib 0.1

Dynamic Selection:

Dynamic Selection (DS) refers to techniques in which the base classifiers are selected on the fly, according to each new sample to be classified. Only the most competent, or an ensemble containing the most competent classifiers is selected to predict the label of a specific test sample. The rationale for such techniques is that not every classifier in the pool is an expert in classifying all unknown samples; rather, each base classifier is an expert in a different local region of the feature space.

DS is one of the most promising MCS approaches due to the fact that more and more works are reporting the superior performance of such techniques over static combination methods. Such techniques have achieved better classification performance especially when dealing with small-sized and imbalanced datasets.

Installation:

The package can be installed using pip:

Stable version:

pip install deslib

Latest version (under development):

pip install git+https://github.com/Menelau/DESlib

Dependencies:

DESlib is tested to work with Python 3.5, and 3.6. The dependency requirements are:

• scipy(>=0.13.3)

• numpy(>=1.10.4)

• scikit-learn(>=0.19.0)

These dependencies are automaticatically installed using the pip commands above.

Description:

The library is divided into four modules:

1. deslib.des: Implementation of DES techniques.

2. deslib.dcs: Implementation of DCS techniques.

3. deslib.static: Implementation of baseline ensemble methods.

4. deslib.util: A collection of aggregation functions and diversity measures for ensemble of classifiers.

• DES techniques currently available are:

  1. META-DES [7] [8] [15]

  2. K-Nearest-Oracle-Eliminate (KNORA-E) [3]

  3. K-Nearest-Oracle-Union (KNORA-U) [3]

  4. Dynamic Ensemble Selection-Performance(DES-P) [12]

  5. K-Nearest-Output Profiles (KNOP) [9]

  6. Randomized Reference Classifier (DES-RRC) [10]

  7. DES Kullback-Leibler Divergence (DES-KL) [12]

  8. DES-Exponential [21]

  9. DES-Logarithmic [11]

  10. DES-Minimum Difference [21]

  11. DES-Clustering [16]

  12. DES-KNN [16]

• DCS techniques currently available are:

  1. Modified Classifier Rank (Rank) [19]

  2. Overall Locall Accuracy (OLA) [4]

  3. Local Class Accuracy (LCA) [4]

  4. Modified Local Accuracy (MLA) [23]

  5. Multiple Classifier Behaviour (MCB) [5]

  6. A Priori Selection (A Priori) [6]

  7. A Posteriori Selection (A Posteriori) [6]

• Baseline methods:

  1. Oracle [20]

  2. Single Best [2]

  3. Static Selection [2]

Variations of each DES techniques are also provided by the library (e.g., different versions of the META-DES framework). Moreover, each DES technique can be used as Dynamic Selection, Dynamic Weighting or a Hybrid version which includes dynamic selection + weighting. The library also offers the implementation of methods which can be applied to DS techniques in order to improve the classification accuracy such as the Dynamic Frienemy Pruning (DFP) [13].

Examples:

Here we show an example using the KNORA-E method using random forest as a pool of classifiers:

from sklearn.ensemble import RandomForestClassifier
from deslib.des.knora_e import KNORAE


# Train a pool of 10 classifiers
pool_classifiers = RandomForestClassifier(n_estimators=10)
pool_classifiers.fit(X_train, y_train)

# Initialize the DES model
knorae = KNORAE(pool_classifiers)

# Preprocess the Dynamic Selection dataset (DSEL)
knorae.fit(X_dsel, y_dsel)

# Predict new examples:
knorae.predict(X_test)

The library accepts any list of classifiers (from scikit-learn) as input, including a list containing different classifier models (heterogeneous ensembles). More examples to use the API can be found in the documentation and im the Examples directory.

References: