Manage multiple configuration sources in a single place
Project description
===============================
Etcaetera
===============================
.. image:: https://badge.fury.io/py/etcaetera.png
:target: http://badge.fury.io/py/etcaetera
.. image:: https://travis-ci.org/oleiade/etcaetera.png?branch=master
:target: https://travis-ci.org/oleiade/etcaetera
.. image:: https://pypip.in/d/etcaetera/badge.png
:target: https://crate.io/packages/etcaetera?version=latest
What?
=====
Etcaetera is the simplest way for you to load your application configuration from multiple sources.
It exposes a single **Config** object to which you add prioritized sources adapters (env, files, cmdline, modules...).
Once you call ``load`` method on it: your settings are loaded from your adapters in the right order, all your configuration is stored in the **Config** object.
You're **done**.
Why?
====
Managing a large application configuration sources can be a pain in the neck.
Command line, files, system environment, modules, the settings you seek come from a lot of mixed sources.
They all have a different accessing mode and merging them can sometimes seem impossible !
Etcaetera provides you with a simple and unified way to handle all the complexity in a single place.
Installation
============
With pip
--------
.. code-block:: bash
Usage
=====
Dive
----
A real world example is worth a thousand words
.. code-block:: python
>>> import os
>>> from etcaetera.config import Config
>>> from etcaetera.adapter import Defaults, Module, Overrides, Env, File
# Let's create a new configuration object
>>> config = Config()
# And create a bunch of adapters
>>> env_adapter = Env("MY_FIRST_SETTING", "MY_SECOND_SETTING")
>>> python_file_adapter = File('/etc/my/python/settings.py')
>>> json_file_adapter = File('/etc/my_json_settings.json')
>>> module_adapter = Module(os)
>>> overrides = Overrides({"MY_FIRST_SETTING": "my forced value"})
# Let's register them
>>> config.register(env_adapter, python_file_adapter, json_file_adapter, module_adapter, overrides)
# Load configuration
>>> config.load()
# And that's it
>>> print(config)
{
"MY_FIRST_SETTING": "my forced value",
"MY_SECOND_SETTING": "my second value",
"FIRST_YAML_SETTING": "first yaml setting value found in yaml settings",
"FIRST_JSON_SETTING": "first json setting value found in json settings",
...
}
Config object
-------------
The config object is the central place for your whole application settings. It will load your adapters
in the order you've registered them, and update itself using it's data. Furthermore you can attach sub config
objects to it, in order to keep a clean configuration hierarchy.
Please note that **Defaults** adapter will always be loaded first, and **Overrides** will always be loaded last.
.. code-block:: python
>>> from etcaetera.config import Config
>>> from etcaetera.adapter import Defaults, Env
# Create a Config object
>>> config = Config()
# Let's register adapters to it. When adapters are registered on a Config
# they are not immediately evaluated.
>>> config.register(Defaults({"ABC": "123"}))
>>> assert "ABC" not in config
>>> config.register(Env("USER", "PWD")
>>> assert "USER" not in config
>>> assert "PWD" not in config
# We can see that our adapters registration has been taken in account
>>> config.adapters
AdapterSet(<Defaults 0x1238f2a>, <Env 0xe3a12bd>)
# Whenever you call load, adapters are evaluated in the order you've
# registered them, and your config values are updated accordingly
>>> config.load()
>>> print(config)
{
"ABC": "123",
"USER": "your user",
"PWD": "/current/working/directory"
}
# If you need a certain hierarchy for your configuration
# Config objects supports sub configs. Here's an example of
# how to add an "aws" subconfig
>>> aws_config = Config() # Create a config obj
>>> aws_env = Env("AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY")
>>> aws_config.register(aws_env) # Register an env adapter on to it
>>> config.add_subconfig('aws', aws_config)
>>> config.aws
{
"AWS_ACCESS_KEY_ID": "128u09ijod019jhd182o1290d81",
"AWS_SECRET_ACCESS_KEY": "qoiejdn0182hern1d098uj12podij1029udaiwjJBIU09u0oimJHKI"
}
# Config objects are also able to automatically format the keys incoming
# from adapters. For example if you'd want all your keys to be lowercased
# just pass it the appropriate formatter (from etcaetera.formatters import *)
>>> from etcaetera.formatters import lowercased
>>> formatted_config = Config(formatter=lowercased)
>>> env_adapter = Env(**{"USER": "SUPER_DUPER_USER"})
>>> formatted_config.register(env_adapter)
>>> formatted_config.load()
>>> print formatted_config
{
"super_duper_user": "oleiade",
}
Adapters
--------
Adapters are the interfaces with configuration sources. They load settings from their custom source type,
and they expose them as a normalized dict to *Config* objects.
Right now, etcaetera provides the following adapters:
* *Defaults*: sets some default settings
* *Overrides*: overrides the config settings values
* *Env*: extracts configuration values from system environment
* *File*: extracts configuration values from a file. Accepted format are: json, yaml, python module file (see *File adapter* section for more details)
* *Module*: extracts configuration values from a python module. Like in django, only uppercased variables will be matched
In a close future, etcaetera may provide adapters for:
* *Argv* argparse format support: would load settings from an argparser parser attributes
* *File* ini format support: would load settings from an ini file
Cool features you should know about:
* You can provide a *formatter* to your adapters so the imported keys will be automatically modified. Example ``Env("USER", etcaetera.formatters.lowercased)`` will import the ``
FIRST_SETTING = 123
SECOND_SETTING = "this is the second value"
THIRD_SETTING = {"easy as": "do re mi"}
ignored_value = "this will be ignore"
*File adapter output will look like this*:
.. code-block:: python
>>> from etcaetera.adapter import File
>>> file = File('/my/settings.py')
>>> file.load()
>>> print(file.data)
{
"FIRST_SETTING": 123,
"SECOND_SETTING": "this is the second value",
"THIRD_SETTING": {"easy as": "do re mi"}
}
Serialized files (aka json and yaml)
````````````````````````````````````
*Given the following json file content*:
.. code-block:: bash
$ cat /my/json/file.json
{
"FIRST_SETTING": "first json file extracted setting",
"SECOND_SETTING": "second json file extracted setting"
}
*The File adapter output will look like this*:
.. code-block:: python
>>> from etcaetera.adapter import File
# The File adapter awaits on a file path at construction.
# All you have to do then, is to let the magic happen
>>> file = File('/my/json/file.json')
>>> file.load()
>>> print(file.data)
{
"FIRST_SETTING": "first json file extracted setting",
"SECOND_SETTING": "second json file extracted setting"
}
Module adapter
~~~~~~~~~~~~~~
The Module adapter will load settings from a python module. It emulates the django
settings module loading behavior, so that every uppercased locals of the module is matched.
**Given a mymodule.settings module looking this**:
.. code-block:: python
MY_FIRST_SETTING = 123
MY_SECOND_SETTING = "abc"
**Loaded module data will look like this**:
.. code-block:: python
>>> import mymodule
>>> from etcaetera.adapter import Module
# It will extract all of the module's uppercased local variables
>>> module = Module(mymodule.settings)
>>> module.load()
>>> print(module.data)
{
MY_FIRST_SETTING = 123
MY_SECOND_SETTING = "abc"
}
Contribute
==========
Please read the `Contributing <https://github.com/oleiade/etcaetera/blob/develop/CONTRIBUTING.rst>`_ instructions
If you are lazy, here's a summary:
1. Found a bug? Want to add a feature? Check for open issues or open a fresh one to start a discussion about it.
2. Fork the repository, and start making your changes.
3. Write some tests showing you fixed the current bug or your feature works as expected.
4. Fasten your seatbelt, and send a pull request to the *develop* branch.
License
=======
The MIT License (MIT)
Copyright (c) 2014 Théo Crevon
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Credits and thanks
==================
Etcaetera is freely inspired (and could be considered as a port of) by [Conf](https://github.com/jingweno/conf).
First of all thanks to it's author. And thanks to @botify who supported the creation of this open-source library.
.. image:: https://d2weczhvl823v0.cloudfront.net/oleiade/etcaetera/trend.png
:alt: Bitdeli badge
:target: https://bitdeli.com/free
History
-------
0.1.0 (2014-01-11)
++++++++++++++++++
* First release on PyPI.
Etcaetera
===============================
.. image:: https://badge.fury.io/py/etcaetera.png
:target: http://badge.fury.io/py/etcaetera
.. image:: https://travis-ci.org/oleiade/etcaetera.png?branch=master
:target: https://travis-ci.org/oleiade/etcaetera
.. image:: https://pypip.in/d/etcaetera/badge.png
:target: https://crate.io/packages/etcaetera?version=latest
What?
=====
Etcaetera is the simplest way for you to load your application configuration from multiple sources.
It exposes a single **Config** object to which you add prioritized sources adapters (env, files, cmdline, modules...).
Once you call ``load`` method on it: your settings are loaded from your adapters in the right order, all your configuration is stored in the **Config** object.
You're **done**.
Why?
====
Managing a large application configuration sources can be a pain in the neck.
Command line, files, system environment, modules, the settings you seek come from a lot of mixed sources.
They all have a different accessing mode and merging them can sometimes seem impossible !
Etcaetera provides you with a simple and unified way to handle all the complexity in a single place.
Installation
============
With pip
--------
.. code-block:: bash
Usage
=====
Dive
----
A real world example is worth a thousand words
.. code-block:: python
>>> import os
>>> from etcaetera.config import Config
>>> from etcaetera.adapter import Defaults, Module, Overrides, Env, File
# Let's create a new configuration object
>>> config = Config()
# And create a bunch of adapters
>>> env_adapter = Env("MY_FIRST_SETTING", "MY_SECOND_SETTING")
>>> python_file_adapter = File('/etc/my/python/settings.py')
>>> json_file_adapter = File('/etc/my_json_settings.json')
>>> module_adapter = Module(os)
>>> overrides = Overrides({"MY_FIRST_SETTING": "my forced value"})
# Let's register them
>>> config.register(env_adapter, python_file_adapter, json_file_adapter, module_adapter, overrides)
# Load configuration
>>> config.load()
# And that's it
>>> print(config)
{
"MY_FIRST_SETTING": "my forced value",
"MY_SECOND_SETTING": "my second value",
"FIRST_YAML_SETTING": "first yaml setting value found in yaml settings",
"FIRST_JSON_SETTING": "first json setting value found in json settings",
...
}
Config object
-------------
The config object is the central place for your whole application settings. It will load your adapters
in the order you've registered them, and update itself using it's data. Furthermore you can attach sub config
objects to it, in order to keep a clean configuration hierarchy.
Please note that **Defaults** adapter will always be loaded first, and **Overrides** will always be loaded last.
.. code-block:: python
>>> from etcaetera.config import Config
>>> from etcaetera.adapter import Defaults, Env
# Create a Config object
>>> config = Config()
# Let's register adapters to it. When adapters are registered on a Config
# they are not immediately evaluated.
>>> config.register(Defaults({"ABC": "123"}))
>>> assert "ABC" not in config
>>> config.register(Env("USER", "PWD")
>>> assert "USER" not in config
>>> assert "PWD" not in config
# We can see that our adapters registration has been taken in account
>>> config.adapters
AdapterSet(<Defaults 0x1238f2a>, <Env 0xe3a12bd>)
# Whenever you call load, adapters are evaluated in the order you've
# registered them, and your config values are updated accordingly
>>> config.load()
>>> print(config)
{
"ABC": "123",
"USER": "your user",
"PWD": "/current/working/directory"
}
# If you need a certain hierarchy for your configuration
# Config objects supports sub configs. Here's an example of
# how to add an "aws" subconfig
>>> aws_config = Config() # Create a config obj
>>> aws_env = Env("AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY")
>>> aws_config.register(aws_env) # Register an env adapter on to it
>>> config.add_subconfig('aws', aws_config)
>>> config.aws
{
"AWS_ACCESS_KEY_ID": "128u09ijod019jhd182o1290d81",
"AWS_SECRET_ACCESS_KEY": "qoiejdn0182hern1d098uj12podij1029udaiwjJBIU09u0oimJHKI"
}
# Config objects are also able to automatically format the keys incoming
# from adapters. For example if you'd want all your keys to be lowercased
# just pass it the appropriate formatter (from etcaetera.formatters import *)
>>> from etcaetera.formatters import lowercased
>>> formatted_config = Config(formatter=lowercased)
>>> env_adapter = Env(**{"USER": "SUPER_DUPER_USER"})
>>> formatted_config.register(env_adapter)
>>> formatted_config.load()
>>> print formatted_config
{
"super_duper_user": "oleiade",
}
Adapters
--------
Adapters are the interfaces with configuration sources. They load settings from their custom source type,
and they expose them as a normalized dict to *Config* objects.
Right now, etcaetera provides the following adapters:
* *Defaults*: sets some default settings
* *Overrides*: overrides the config settings values
* *Env*: extracts configuration values from system environment
* *File*: extracts configuration values from a file. Accepted format are: json, yaml, python module file (see *File adapter* section for more details)
* *Module*: extracts configuration values from a python module. Like in django, only uppercased variables will be matched
In a close future, etcaetera may provide adapters for:
* *Argv* argparse format support: would load settings from an argparser parser attributes
* *File* ini format support: would load settings from an ini file
Cool features you should know about:
* You can provide a *formatter* to your adapters so the imported keys will be automatically modified. Example ``Env("USER", etcaetera.formatters.lowercased)`` will import the ``
FIRST_SETTING = 123
SECOND_SETTING = "this is the second value"
THIRD_SETTING = {"easy as": "do re mi"}
ignored_value = "this will be ignore"
*File adapter output will look like this*:
.. code-block:: python
>>> from etcaetera.adapter import File
>>> file = File('/my/settings.py')
>>> file.load()
>>> print(file.data)
{
"FIRST_SETTING": 123,
"SECOND_SETTING": "this is the second value",
"THIRD_SETTING": {"easy as": "do re mi"}
}
Serialized files (aka json and yaml)
````````````````````````````````````
*Given the following json file content*:
.. code-block:: bash
$ cat /my/json/file.json
{
"FIRST_SETTING": "first json file extracted setting",
"SECOND_SETTING": "second json file extracted setting"
}
*The File adapter output will look like this*:
.. code-block:: python
>>> from etcaetera.adapter import File
# The File adapter awaits on a file path at construction.
# All you have to do then, is to let the magic happen
>>> file = File('/my/json/file.json')
>>> file.load()
>>> print(file.data)
{
"FIRST_SETTING": "first json file extracted setting",
"SECOND_SETTING": "second json file extracted setting"
}
Module adapter
~~~~~~~~~~~~~~
The Module adapter will load settings from a python module. It emulates the django
settings module loading behavior, so that every uppercased locals of the module is matched.
**Given a mymodule.settings module looking this**:
.. code-block:: python
MY_FIRST_SETTING = 123
MY_SECOND_SETTING = "abc"
**Loaded module data will look like this**:
.. code-block:: python
>>> import mymodule
>>> from etcaetera.adapter import Module
# It will extract all of the module's uppercased local variables
>>> module = Module(mymodule.settings)
>>> module.load()
>>> print(module.data)
{
MY_FIRST_SETTING = 123
MY_SECOND_SETTING = "abc"
}
Contribute
==========
Please read the `Contributing <https://github.com/oleiade/etcaetera/blob/develop/CONTRIBUTING.rst>`_ instructions
If you are lazy, here's a summary:
1. Found a bug? Want to add a feature? Check for open issues or open a fresh one to start a discussion about it.
2. Fork the repository, and start making your changes.
3. Write some tests showing you fixed the current bug or your feature works as expected.
4. Fasten your seatbelt, and send a pull request to the *develop* branch.
License
=======
The MIT License (MIT)
Copyright (c) 2014 Théo Crevon
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Credits and thanks
==================
Etcaetera is freely inspired (and could be considered as a port of) by [Conf](https://github.com/jingweno/conf).
First of all thanks to it's author. And thanks to @botify who supported the creation of this open-source library.
.. image:: https://d2weczhvl823v0.cloudfront.net/oleiade/etcaetera/trend.png
:alt: Bitdeli badge
:target: https://bitdeli.com/free
History
-------
0.1.0 (2014-01-11)
++++++++++++++++++
* First release on PyPI.
