Domain-Model-eXaminer (DMX)

Description

The goal of this process is to perform machine reading over the model codebase in order to automatically extract key metadata about:

• Input files
• Output files
• System requirements
• Software requirements
• Model entry points
• Model descriptions
• Maintainer information

Installation

Pip package

The domain-model-examiner library can be installed via pip:

pip install domainmodelexaminer

Local installation from source

After cloning the repo, the recommended installation is to create a new conda environment and then install from requirements.txt:

conda create --name=dmx python=3.8
conda activate dmx
conda install -c conda-forge uvicorn
pip install -r requirements.txt

Docker installation from source

After cloining the repo, docker install and run:

docker build -t dmx .
docker run -p 80:80 dmx

The API docs will be available at http://0.0.0.0/docs.

Usage

Pip Package

The examine function of module dmx of package domainmodelexaminer takes the following parameters:

• url (str) GitHub or GitLab repo url.
• return_json (bool = True) Returns a JSON string if true, else a YAML string.

and returns either JSON (default) or a YAML string.

The examine function can be imported an executed as follows:

from domainmodelexaminer import dmx
dmx_json_output = dmx.examine("https://github.com/dojo-modeling/dojo")

Local install

List command line options

python dmx.py --help

Process Local Repository

Clone or download the repository of interest. For example:

cd /tmp
git clone https://github.com/jataware/dummy-model.git

Next, analyze this repo with:

python dmx.py --repo="/tmp/dummy-model"

Process Remote Repository

python dmx.py --url="https://github.com/jataware/dummy-model.git"

This creates and deletes a tempory folder 'tmp'.

Output

The application produces a .yaml file with prefix 'dmx-' concatenated with the repo name.

Running the API

Run API:

uvicorn domainmodelexaminer.api:app

The API docs will be available at http://127.0.0.1:8000/docs

Do not use the uvicorn --reload flag, since the app will download the repo to the api folder.

Testing Repositories

• https://github.com/jataware/dummy-model
• https://github.com/djgroen/FabFlee
• https://github.com/jataware/maxhop
• https://github.com/DSSAT/pythia/tree/develop [develop branch]
• https://github.com/mjpuma/FSC-WorldModelers

Note that Pythia is a highly abstracted Python version of the DSSAT model.

API Reference

Default port for Docker is 80. Default port for Uvicorn is 8000. Examples given below assume the Uvicorn server is running at port 8000.

Docs

  GET /docs
  e.g. http://127.0.0.1:8000/docs

Confirm the API is available:

  GET /
  e.g. http://127.0.0.1:8000/

Returns

{"status":"running"}

Examine URL

  GET /examine/${url}
  e.g. http://127.0.0.1:8000/examine/?url=https://github.com/jataware/dummy-model.git

Returns

JSON or YAML-formatted string e.g.

{
  "language": "Python",
  "owner": {
    "login": "dojo-modeling",
    "repo_url": "https://github.com/dojo-modeling/dojo",
    "type": "Organization",
    "name": null,
    "company": null,
    "blog": "",
    "location": null,
    "bio": null
  },
  "about": "not found",
  "docker_entrypoint": null,
  "model_types": [
    "Geospatial Data / Terrain Models"
  ],
  "imports": [
  ...