datapackage-pipelines-goodtables 0.0.2a0

Install

# clone the repo and install it with pip

git clone https://github.com/frictionlessdata/datapackage-pipelines-goodtables.git
pip install -e .

Usage

Add the following to the pipeline-spec.yml configuration to validate each resource in the datapackage. A report is outputted to the logger.

...
- run: goodtables.validate
  parameters:
      fail_on_error: True,
      reports_path: 'path/to/datapackage/reports',  # where reports will be written
      datapackage_reports_path: 'reports',  # relative to datapackage.json
      write_report: True,
      goodtables:
          <key>: <value>  # options passed to goodtables.validate()

• fail_on_error: An optional boolean to determine whether the pipeline should fail on validation error (default True).

• reports_path: An optional string to define where Goodtables reports should be written (default is reports).

• datapackage_reports_path: An optional string to define the path to the report, relative to the datapackage.json (see note below).

• write_report: An optional boolean to determine whether a goodtables validation report should be written to reports_path (default is True).

• goodtables: An optional object passed to goodtables.validate() to customise its behaviour. See `goodtables.validate() <https://github.com/frictionlessdata/goodtables-py/#validatesource-options>`__ for available options.

If reports are written, and datapackage_reports_path is defined, a reports property will be added to the datapackage, detailing the path to the report for each resource:

...
"reports": [
    {
        "resource": "my-resource",
        "reportType": "goodtables",
        "path": "path/to/my-resource.json"
    }
]

It is recommended that datapackage_reports_path is used to define a relative path, from the datapackage.json file, that represents where the report was written. datapackage_reports_path does not define where the reports will be written, but helps ensure a correct path is defined in the reports property in datapackage.json. This is useful when the pipeline concludes with a dump_to.path processor.