Metaflow library for SAP AI Core

This Python library is a plugin that lets you generate workflow templates that can be executed in Argo or on SAP AI Core.

Sap-ai-core-metaflow is a Metaflow plugin to execute flows on Argo Workflows.

It adds Argo Workflows capabilities on top of Metaflow's @kubernetes decorator similar to step-functions and @batch.

Installation

pip install sap-ai-core-metaflow\[kubernetes\]

Documentation

For general information about how to use Metaflow, see the Metaflow documentation https://docs.metaflow.org.

For command-line help, use the command option --help

Configuration

The plugin requires configured Metaflow:

metaflow configure eks

Usage

To see a sub-commands' detailed information, run::

python 00-helloworld/helloworld.py argo [sub-command] --help

An example creating an argo workflow from the Metaflow hello world tutorial.

metaflow tutorials pull
cd metaflow-tutorials

python 00-helloworld/helloworld.py argo create
python 00-helloworld/helloworld.py argo trigger
python 00-helloworld/helloworld.py argo list-runs

Generate Argo Workflows Template

To store a generated template in a file, run the command:

python 00-helloworld/helloworld.py --with=kubernetes:secrets=default-object-store-secret argo create --only-json > helloworld.json

Note

Argo Workflows accepts both YAML and JSON templates. The Argo plugin always creates JSON.