Twinn-ml-interface

Twinn-ml-interface is a Python package for data contracts between machine learning code and infrastructure.

Author: Royal HaskoningDHV

Installation

The easiest way to install is package is using pip:

pip install twinn-ml-interface

Model Interface

Purpose

The Model Interface defines the required methods and attributes that any ML model needs to have in order to run in our infrastructure.

Testing compliance of your model with the data contract

Instance of the Model Interface

Once all the attributes and methods from the Protocol ModelInterfaceV4 are implemented, including the correct type-hints / annotations, we can check if our models is compliant with the interface if it passes the isinstance check with ModelInterfaceV4. You can find a base test in twinn_ml_interface/interface/model_test.py. The Darrow-Poc is an example of a model that follows the ModelInterfaceV4.

Mock Executors

The executor class takes care of running the model either for training or predictions in our infrastructure. Here, we implemented a mock executor to emulate that behaviour to some extent, which hopefully makes it a little clearer in what context the model class will be used. Any model compliant with the ModelInterface should be able to train and predict using the ExecutorMock that can be found in twinn_ml_interface/mocks/mocks.py. The Darrow-Poc is an example of a model that follows ModelInterfaceV4 and, for instance, can run using the ExecutorMock.

The steps and methods that the infrastructure and the mock executor run during training are:

1. Read config:
   • get_target_template()
   • get_train_window_finder_config_template()
2. Initialize the model
   • initialize()
3. Given the configuration for the train window finder in the previous steps, validate possible windows:
   • validate_input_data()
4. Read the data configuration to download all the needed data in a window selected by the previous step:
   • get_data_config_template()
5. Transform the input data as needed:
   • preprocess()
6. Train:
   • train()
7. Store the model:
   • dump()

When the training is finished, the model can be used for predicting. The prediction steps are:

1. Retrieve the model from storage and load it:
   • load()
2. Fetch the data needed for prediction based on either:
   • base_features - if present
   • get_data_config_template() - otherwise
3. Predict:
   • predict()
4. Load configuration to post predictions:
   • get_result_template()

Example of the Model Interface

Darrow Poc

The Darrow-Poc is an example of a model that follows ModelInterfaceV4.