yschema 1.0.0

YSchema
=======

YSchema is a terse and simple schema format with a reference validator
implementation for YAML, JSON and other dictionary based data structures.

YSchema is quite minimal (in terms of lines of code) and is continuously tested
against a set of of valid and invalid example data (see the ``examples``
directory). The code works nicely for its intended purpose, but may not be the
most powerful or popular, even if it does what it was intended for very well.
The main assumption (at least for now) is that all keys are strings without
whitespace.

YSchema is written in Python (v. 3) and validates dictionaries containing basic
datatypes like strings, ints, floats, lists and nested dictionaries. The schema
is also a dictionary, so both the data and the schema can be written in Python,
JSON, YAML, TOML, ... formats. YSchema cannot validate all possible YAML / JSON
data, in fact it cannot even validate its own schema files since those use
significant white space in dictionary keys to describe expected data types and
whether the data is required or not.

To install the YSchema Python library along with the ``yschema`` command line
program run:

.. code:: bash

python3 -m pip install -U yschema

Consider using a virtual environment or adding ``--user`` to the ``pip`` command
if you do not want to install into the system's site-packages directory. PS: You
may also want to look at older and more established schema and validators such
as Yamale_ or json-schema_ in case those serve your needs better.

.. _Yamale: https://github.com/23andMe/Yamale
.. _json-schema: http://json-schema.org


.. contents::


Introduction to YSchema
-----------------------

A simple example schema:

.. code:: yaml

# There must be a key "name" that maps to a string
required name: str

# There can be an integer age, but it is not required
optional age: int

# The optional height must be above 0
optional height: float(min_val=0)

To validate this, first load the schema above into a dictionary, then load the
data to validate into another dictionary, and finally run:

.. code:: python

import yschema

# possibly loaded from json or yaml or just a plain old dict
schema = my_load_schema_function()
data_dict = {'name': 'Tormod'}

yschema.validate(data_dict, schema_dict)

If the function does not raise ``yschema.ValidationError`` then the data is
valid according to the given schema. You can also use the ``yschema`` command
to validate YAML files from the command line.

A more complicated example, showing constants and nested dictionaries:

.. code:: yaml

# Example of a constant that can be used in validation functions
constant minimum_string_length: 5

# A sub-dictionary
type Whale:
# The name is a string of a given minimum length
required name: str(min_len=minimum_string_length)

# The length must be between 0 and 500 meters
optional length: float(min_val=0, max_val=500.0)

required whales: list(type=Whale)

The above schema validates data like this:

.. code:: yaml

whales:
- name: Unknown Whale
- name: Enormous Whale
length: 200.0

Note that when working with aliases and types the order of the keys in the
dictionary starts to matter. Either use a Python 3.6 or later, or load your
data into an OrderedDict. For PyYAML this means:

.. code:: python

from collections import OrderedDict
import yaml
yaml.add_constructor(yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
lambda loader, node: OrderedDict(loader.construct_pairs(node)))

More advanced features
----------------------

**Built in types**: the following types are implemented. Optional parameters
are listed below each type:

* Any
* bool
* str
- min_len
- max_len
- equals - e.g. ``str(equals='Hi!')`` or matching one of several
pissibilities with ``str(equals=('a', 'b', 'c'))``
- prefix
* int
- min_val
- max_val
- equals - e.g. ``int(equals=3)`` or ``int(equals=(2, 4, 6))``
* float
- min_val
- max_val
- equals - e.g. ``float(equals=3.2)`` or ``float(equals=(2.1, 4.4))``
* list
- min_len
- max_len
- type - e.g. ``list(type=int)`` or ``list(type=Whale)``
* one_of
- types - e.g. ``one_of(types=(int, str))`` or
``one_of(types=(str(prefix='Moby'), Whale))``
* any_of
- types - see ``one_of`` (``any_of`` matches if any of the types match,
``one_of`` requires exactly one match)

**Alias**: you can give an alias to avoid typing the same type definition over
and over again:

.. code:: yaml

alias Cat: one_of(types=(HouseCat, Tiger, Lynx))
alias Cats: list(type=Cat)

**Glob**: you can allow undefined keys by using a glob. The following will
validate OK for all documents

.. code:: yaml

optional *: Any

**Inherit**: a sub-schema introduced by ``type`` can contain a key ``inherit``
with the name of a previously defined sub-schema to avoid repeating
definitions that are shared among several types:

.. code:: yaml

type MeshBase:
optional move: list(type=str)
optional sort_order: list(type=int)
optional mesh_file: str
type MeshDolfinFile:
inherit: MeshBase
required type: str(equals=('XML', 'XDMF', 'HDF5'))
required mesh_file: str
optional facet_region_file: str
type MeshMeshio:
inherit: MeshBase
required type: str(equals='meshio')
required mesh_file: str
optional meshio_type: str
required mesh: one_of(types=(MeshMeshio, MeshDolfinFile))


Releases
--------

Version 1.0.0 - June 7. 2018
............................

Completed v 1.0 implementation goals. The YSchema language is powerful enough to
express most of what I wanted for validating Ocallaris_ input files. The code
base is decently tested (using the fantastic CircleCI service) and a command
line tool is also included for validating YAML files from the shell.

There may not be a large number of additional releases if no more features are
found to be necessary for the author's uses. It is relatively easy to add new
type validators from user code, but feel free to submit a pull request if you
are finding YSchema useful and have implemented some general purpose validators.
YSchema does not intend to compete with complex and more fully featured schema
languages like json-schema_.

.. _Ocellaris: https://bitbucket.org/trlandet/ocellaris

Copyright and license
---------------------

YSchema is copyright Tormod Landet, 2018. YSchema is licensed under the Apache
2.0 license, a permissive free software license compatible with version 3 of
the GNU GPL. See the file LICENSE for the details.