drs-compliance-suite

Tests to verify the compliance of a DRS implementation with GA4GH Data Repository Service (DRS) specification. This compliance suite currently supports the following DRS versions and will aim to support more versions of DRS in the future.

• DRS 1.2.0

Installations

• Python 3.x is required to run DRS Compliance Suite natively or using PyPI package.
• Docker Desktop is required to run DRS Compliance Suite using a docker image.

Running DRS Compliance Suite

1. Natively

Install the packages from requirements.txt

cd drs-compliance-suite
pip3 install -r requirements.txt

Add PYTHONPATH to env variables

export PYTHONPATH=<absolute path to drs-compliance-suite>

Run the compliance suite

python3 compliance_suite/report_runner.py --server_base_url "http://localhost:8085/ga4gh/drs/v1" --platform_name "ga4gh starter kit drs" --platform_description "GA4GH reference implementation of DRS specification" --drs_version "1.2.0" --config_file "compliance_suite/config/config_samples/config_basic.json" --serve --serve_port 56565

Note: This specific command is an example of running the compliance suite on a local deployment of DRS that is running on port 8085.
When running the compliance suite, it's important to configure the command line arguments according to the specific DRS implementation you're testing. Please refer to the Command Line Arguments section for details on each of these arguments.

2. Using PyPI Package

Install the latest version of the drs-compliance-suite PyPI package using pip3

pip3 install drs-compliance-suite --upgrade

Run the compliance suite

drs-compliance-suite --server_base_url "http://localhost:8085/ga4gh/drs/v1" --platform_name "ga4gh starter kit drs" --platform_description "GA4GH reference implementation of DRS specification" --drs_version "1.2.0" --config_file "compliance_suite/config/config_samples/config_basic.json" --serve --serve_port 56565

Note: This specific command is an example of running the compliance suite on a local deployment of DRS that is running on port 8085.
When running the compliance suite, it's important to configure the command line arguments according to the specific DRS implementation you're testing. Please refer to the Command Line Arguments section for details on each of these arguments.

3. Using Docker

Pull the latest docker image from dockerhub.

docker pull ga4gh/drs-compliance-suite:1.0.3

Run the compliance suite using the docker image

docker run -d --name drs-compliance-suite -v $(PWD)/output/:/usr/src/app/output/ -p 57568:57568 ga4gh/drs-compliance-suite:1.0.3: --server_base_url "http://host.docker.internal:8085/ga4gh/drs/v1" --platform_name "ga4gh starter kit drs" --platform_description "GA4GH reference implementation of DRS specification" --report_path "./output/test-report.json" --drs_version "1.2.0" --config_file "compliance_suite/config/config_samples/config_none.json" --serve --serve_port 57568

Note: This specific command is an example of running the compliance suite on a local deployment of DRS that is running on port 8085.
When running the compliance suite, it's important to configure the command line arguments according to the specific DRS implementation you're testing. Please refer to the Command Line Arguments section for details on each of these arguments.

Command Line Arguments

Config File:

The compliance suite is provided with information for testing the DRS server through a user-provided JSON config file. This file includes the following details:

• Authorization information for the service-info endpoint
• A few DRS Object IDs that are present in the DRS server
• Authorization information for each of these DRS objects
• Indication of whether the DRS object is a bundle or a single blob

Here's a template for a config file that can be used to configure these details:

{
  "service_info": {
      "auth_type": "basic",
      "auth_token": "dXNlcm5hbWU6cGFzc3dvcmQ="
  },
  "drs_objects" : [
      {
          "drs_id": "697907bf-d5bd-433e-aac2-1747f1faf366",
          "auth_type": "none",
          "auth_token": "",
          "is_bundle": false
      },
      {
          "drs_id": "0bb9d297-2710-48f6-ab4d-80d5eb0c9eaa",
          "auth_type": "basic",
          "auth_token": "dXNlcm5hbWU6cGFzc3dvcmQ=",
          "is_bundle": false
      },
      {
          "drs_id" : "41898242-62a9-4129-9a2c-5a4e8f5f0afb",
          "auth_type": "bearer",
          "auth_token": "secret-bearer-token-1",
          "is_bundle": true
      },
      {
          "drs_id" : "a1dd4ae2-8d26-43b0-a199-342b64c7dff6",
          "auth_type": "passport",
          "auth_token": "43b-passport-a1d",
          "is_bundle": true
      }
  ]
}

• The "auth_type" specifies the type of authorization, which can be one of the following: ["basic", "bearer", "passport", "none"]
• The "auth_token" field contains the corresponding authorization token value.
• If "auth_type" is set to "basic", the "auth_token" can be created by using base64 encoding.
• If "auth_type" is set to "none", the "auth_token" field should be left blank with a value of "".
• The "is_bundle" flag indicates whether the DRS object is a bundle or a single blob. If set to True, the object is a bundle, and if set to False, the object is a single blob.

You can find some sample config files here

Unittesting

Run the unittests with coverage

pytest --cov=compliance_suite unittests/

Changelog

v1.0.3

• provide flexibility in providing different auth information for drs object and drs access endpoints
• remove incorrect skip status setting

v1.0.2

• Reduce the docker image size by using python:3.11-slim-bullseye instead of python:3

v1.0.1

• Fixed a bug in the docker deployment of DRS Compliance Suite
• Update README documentation

v1.0.0

• DRS Compliance Suite for Data Repository Service v1.2.0