skip to navigation
skip to content

Not Logged In

anyconfig 0.1.0

Generic access to configuration files in some formats

python-anyconfig [1] is a MIT licensed python library provides generic access to configuration files in various formats with configuration merge along with config template and schema validation/generation support.

[1]This name took an example from the ‘anydbm’ python standard library.

Features

python-anyconfig provides very simple and unified APIs to process configuration files in various formats:

  • anyconfig.load() to load configuration files and return a dict-like object represents configuration itself loaded
  • anyconfig.loads() to load configuration from a string just like json.loads does
  • anyconfig.dump() to dump a configuration file from a dict or dict-like object represents configuration
  • anyconfig.dumps() to dump a configuration string from a dict or dict-like object represents configuration
  • anyconfig.validate() to validate configuration loaded with anyconfig.load() with JSON schema [2] (object) also loaded with anyconfig.load(). anyconfig.load() may help loading JSON schema file[s] in any formats anyconfig supports.
  • anyconfig.gen_schema() to generate a JSON schema object for given configuration file[s] to validate it/them later.

It enables to load a configuration file and configuration files in various formats in the same manner, and in some cases, even there is no need to take care of the actual format of configuration file[s] like the followings:

import anyconfig

# Config type (format) is automatically detected by filename (file
# extension) in some cases.
conf1 = anyconfig.load("/path/to/foo/conf.d/a.yml")

# Loaded config data is a dict-like object, for example:
#
#   conf1["a"] => 1
#   conf1["b"]["b1"] => "xyz"
#   conf1["c"]["c1"]["c13"] => [1, 2, 3]

# Or you can specify the format (config type) explicitly if automatic
# detection may not work.
conf2 = anyconfig.load("/path/to/foo/conf.d/b.conf", "yaml")

# Specify multiple config files by the list of paths. Configurations of each
# files are merged.
conf3 = anyconfig.load(["/etc/foo.d/a.json", "/etc/foo.d/b.json"])

# Similar to the above but all or one of config file[s] is/are missing:
conf4 = anyconfig.load(["/etc/foo.d/a.json", "/etc/foo.d/b.json"],
                       ignore_missing=True)

# Specify config files by glob path pattern:
conf5 = anyconfig.load("/etc/foo.d/*.json")

# Similar to the above, but parameters in the former config file will be simply
# overwritten by the later ones:
conf6 = anyconfig.load("/etc/foo.d/*.json", merge=anyconfig.MS_REPLACE)

Also, it can process configuration files which are actually jinja2-based template files:

  • Enables to load a substantial configuration rendered from half-baked configuration template files with given context
  • Enables to load a series of configuration files indirectly ‘include’-d from a/some configuration file[s] with using jinja2’s ‘include’ directive.
In [1]: import anyconfig

In [2]: open("/tmp/a.yml", 'w').write("a: {{ a|default('aaa') }}\n")

In [3]: anyconfig.load("/tmp/a.yml", ac_template=True)
Out[3]: {'a': 'aaa'}

In [4]: anyconfig.load("/tmp/a.yml", ac_template=True, ac_context=dict(a='bbb'))
Out[4]: {'a': 'bbb'}

In [5]: open("/tmp/b.yml", 'w').write("{% include 'a.yml' %}\n")  # 'include'

In [6]: anyconfig.load("/tmp/b.yml", ac_template=True, ac_context=dict(a='ccc'))
Out[6]: {'a': 'ccc'}

And python-anyconfig enables to validate configuration files in various format with using JSON schema like the followings:

# Validate a JSON config file (conf.json) with JSON schema (schema.yaml).
# If validatation suceeds, `rc` -> True, `err` -> ''.
conf1 = anyconfig.load("/path/to/conf.json")
schema1 = anyconfig.load("/path/to/schema.yaml")
(rc, err) = anyconfig.validate(conf1, schema1)  # err is empty if success, rc == 0

# Validate a config file (conf.yml) with JSON schema (schema.yml) while
# loading the config file.
conf2 = anyconfig.load("/a/b/c/conf.yml", ac_schema="/c/d/e/schema.yml")

# Validate config loaded from multiple config files with JSON schema
# (schema.json) while loading them.
conf3 = anyconfig.load("conf.d/*.yml", ac_schema="/c/d/e/schema.json")

# Generate jsonschema object from config files loaded and get string
# representation.
conf4 = anyconfig.load("conf.d/*.yml")
scm4 = anyconfig.gen_schema(conf4)
scm4_s = anyconfig.dumps(scm4, "json")

And in the last place, python-anyconfig provides a CLI tool called anyconfig_cli to process configuration files and:

  • Convert a/multiple configuration file[s] to another configuration files in different formats
  • Get configuration value in a/multiple configuration file[s]
  • Validate configuration file[s] with JSON schema
  • Generate JSON schema for given configuration file[s]
[2]http://json-schema.org

Supported configuration formats

python-anyconfig supports various (configuration) file formats if the required module is available and the corresponding backend is ready to use:

<caption>Supported formats</caption>
Format Type Required Notes
JSON json json (standard lib) or simplejson [3] Enabled by default.
Ini-like ini configparser (standard lib) Enabled by default.
YAML yaml PyYAML [4] Enabled automatically if the requirement is satisfied.
XML xml lxml [5] or ElementTree Ditto.
ConifgObj configobj configobj [6] Ditto.
MessagePack msgpack msgpack-python [7] Ditto.
TOML toml toml [8] Ditto.
BSON bson bson in pymongo [9] Ditto.

The supported formats of python-anyconfig on your system is able to be listed by ‘anyconfig_cli -L’ like this:

$ anyconfig_cli -L
Supported config types: bson, configobj, ini, json, msgpack, toml, xml, yaml
$

or with the API ‘anyconfig.list_types()’ will show them:

In [8]: anyconfig.list_types()
Out[8]: ['bson', 'configobj', 'ini', 'json', 'msgpack', 'toml', 'xml', 'yaml']

In [9]:

It utilizes plugin mechanism provided by setuptools [10] and other formats may be supported by corresponding pluggale backends (see the next sub section also) like Java properties format.

[3]https://pypi.python.org/pypi/simplejson
[4]https://pypi.python.org/pypi/PyYAML
[5]https://pypi.python.org/pypi/lxml
[6]https://pypi.python.org/pypi/configobj
[7]https://pypi.python.org/pypi/msgpack-python
[8]https://pypi.python.org/pypi/toml
[9]https://pypi.python.org/pypi/pymongo
[10]http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-of-services-and-plugins
[11]https://pypi.python.org/pypi/pyjavaproperties

Installation

Requirements

Many runtime dependencies are resolved dynamically and python-anyconfig just disables specific features if required dependencies are not satisfied. Therefore, only python standard library is required to install and use python-anyconfig at minimum.

The following packages need to be installed along with python-anycofig to enable the features.

Feature Requirements Notes
YAML load/dump PyYAML none
ConifgObj load/dump configobj none
MessagePack load/dump msgpack-python none
TOML load/dump toml none
BSON load/dump bson bson from pymongo package may work and bson [12] does not
Template config Jinja2 none
Validation with JSON schema jsonschema [13] Not required to generate JSON schema.
[12]https://pypi.python.org/pypi/bson/
[13]https://pypi.python.org/pypi/jsonschema/

How to install

There is a couple of ways to install python-anyconfig:

  • Binary RPMs:

    If you’re Fedora or Red Hat Enterprise Linux user, you can install RPMs from the copr repository, http://copr.fedoraproject.org/coprs/ssato/python-anyconfig/.

  • PyPI: You can install python-anyconfig from PyPI with using pip:

    $ pip install anyconfig
    
  • Build RPMs from source: It’s easy to build python-anyconfig with using rpm-build and mock:

    $ python setup.py srpm && mock dist/python-anyconfig-<ver_dist>.src.rpm
    

    or:

    $ python setup.py rpm
    

    and install built RPMs.

  • Build from source: Of course you can build and/or install python modules in usual way such like ‘python setup.py bdist’, ‘pip install git+https://github.com/ssato/python-anyconfig/’ and so on.

Help and feedbak

If you have any issues / feature request / bug reports with python-anyconfig, please open an issue ticket on github.com (https://github.com/ssato/python-anyconfig/issues).

The following areas are still insufficient, I think.

  • Make python-anyconfig robust for invalid inputs
  • Make python-anyconfig scaled: some functions are limited by max recursion depth.
  • Documentation:
    • Especially API docs need more fixes and enhancements! CLI doc is non-fulfilling also.
    • English is not my native lang and there are many wrong and hard-to-understand expressions.

Any feedbacks, helps, suggestions are welcome! Please open github issues for these kind of problems also!

Hacking

How to test

Run ‘[WITH_COVERAGE=1] ./pkg/runtest.sh [path_to_python_code]’ or ‘tox’ for tests.

About test-time requirements, please take a look at pkg/test_requirements.txt.

How to write backend plugin modules

Backend class must inherit anyconfig.backend.Parser and need some member variables and method (‘load_impl’ and ‘dumps_impl’ at minimum) implementations.

JSON and YAML backend modules (anyconfig.backend.{json,yaml}_) should be good examples to write backend modules, I think.

Also, please take a look at some example backend plugin modules mentioned in the Supported configuration formats section.

 
File Type Py Version Uploaded on Size
anyconfig-0.1.0.tar.gz (md5) Source 2015-08-15 61KB
  • Downloads (All Versions):
  • 36 downloads in the last day
  • 337 downloads in the last week
  • 3221 downloads in the last month