e2j2 0.3.0

What is e2j2?

e2j2 (environment to jinja2 variables) is a commandline tool which will render jinja2 templates to textfiles. all environment variables can be used in the jinja2 templates, within the environment variables you can use special tags which give you the option to insert json, json file paths, base64 hashes, consul kv keys.

e2j2 can be used within docker containers, or for other simple configuration templating tasks. Within a docker container you can simply add the j2 extention to a configuration file, or to multiple files within one or more folder structures and then run e2j2 as part of your docker entrypoint script.

Example:

lets assume we want to render the following server block in nginx, if we place the server configuration in a nginx include directory for example /etc/nginx/conf.d

server {
  server_name {{ NGINX.server_name }};
  listen 80;
  listen [::]:80;
  error_page 500 502 503 504 /50x.html;

  location / {
    index {{ NGINX.index_page }};
    root {{ NGINX.web_root }};
  }

  location ~ \.php$ {
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    fastcgi_pass unix:{{ NGINX.fpm_socket }};
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    include {{ NGINX.fcgi_params }};
    root {{ NGINX.web_root }};
    try_files $uri =404;
  }
}

if you then set the NGINX environment variable, running e2j2 will render the jinja2 template and place it in the same folder:

~> export NGINX='json:
{
"server_name": "www.myweb.com",
"index_page": "index.php",
"web_root": "/usr/local/www/myweb",
"fcgi_params": "/usr/local/etc/nginx/myweb-fcgi-params",
"fpm_socket": "/var/run/php-fpm/myweb.socket"
}'
~> e2j2

In: .
    rendering: nginx_vhost_config.conf.j2=>done => writing: nginx_vhost_config.conf=>done

~> cat nginx_vhost_config.conf
server {
  server_name www.myweb.com;
  listen 80;
  listen [::]:80;
  error_page 500 502 503 504 /50x.html;

  location / {
    index index.php;
    root /usr/local/www/myweb;
  }

  location ~ \.php$ {
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    fastcgi_pass unix:/var/run/php-fpm/myweb.socket;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    include /usr/local/etc/nginx/myweb-fcgi-params;
    root /usr/local/www/roundcube;
    try_files $uri =404;
  }
}

Environment variable examples:

MYENVVAR='plain environment variable'

This is a plain environment variable

MYFILEVAR=file:./file-example.txt

This is a file example

MYJSONVAR='json:{"key": "json-example"}'

This is a json-example

MYJSONFILEVAR='jsonfile:jsonfile-example.json'

This is a jsonfile example with subkey

export MYBASE64VAR='base64:YmFzZTY0IGV4YW1wbGU='

This is a base64 example

read -d '' CONSUL_CONFIG << EOF
{
   "url": "https://consul.foobar.tld",
   "token": "abcdef01-0123-abcd-1234-0123456789ab"
}
EOF

export MYCONSULVAR='consul:config={"url": "https://consul2.foobar.tld", "token": "012345678-0123-abcd-1234-0123456789ab"}:consulvar"

export MYCONSULVAR='consul:consulvar'

This is a consul example

export MYLIST='list:"first","second","third","fourth"'

"first"
"second"
"third"
"fourth"

export WORDPRESS='json:{"database": {"name": "mydb", "user": "mydb_user", "password": "{{ DBSECRET }}", "host": "localhost"}}'
export DBSECRET='file:./twopass-secret'

// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define( 'DB_NAME', 'mydb' );

/** MySQL database username */
define( 'DB_USER', 'mydb_user' );

/** MySQL database password */
define( 'DB_PASSWORD', 'Db$ecr3t' );

/** MySQL hostname */
define( 'DB_HOST', 'localhost' );

read -d '' VAULT_CONFIG << EOF
 {
   "url": "https://vault.foobar.tld:8200",
   "token": "s.xxxxxxxxxxxxxxxxxxxxxxx",
   "backend: "kv2"
 }
EOF

export MYVAULTVAR='vault:config={"backend": "kv1"}:kv/my-secret"

vault kv put secret/my-secret secret=topsecret
export MYVAULTVAR='vault:secret/my-secret'

** topsecret **
This is a vault example

Breaking change:

• envvar tag errors will result in undefined variables previously they contained an error message

0.2.1 (2019-10-23)

Added

• add –stacktrace flag to include a python stack trace in the error file (thanks: Sylvia van Os/TheLastProject)

Changed - show jinja2 render error in console output (thanks: Marco Verleun)

Fixed - Fix for pip install issues - Pass exit code to calling shell

0.2.0 (2019-07-23)

Added

• vault secrets support

• per tag config (for the consul and vault tags)

• add support for token variables (for the consul and vault tags)

Changed

• Add json schema validation for consul and vault config

0.1.21 (2019-07-09)

Fixed

• Fix issue with included templates in other directories

Added

• Add option to “copy” file ownership and file permissions

0.1.20 (2019-06-12)

Fixed

• fix “only render one file” issue

0.1.19 (2019-06-10)

Added

• add option to whitelist / blacklist environment variables

Fixed

• set proper exit codes (0=success / 1=failure)

Changed

• add unittests

• add functional tests

0.1.18 (2019-05-17)

Fixed

• Fix bug in consul parser

0.1.17 (2019-05-17)

Fixed

• Fix bug with include statements in jinja2 template

0.1.16 (2019-05-16)

Fixed

• Fix bug with default jinja marker

0.1.15 (2019-05-16)

Changed

• add options to overwrite the default jinja2 markers (block start/end, variable start/end and comment start/end)

0.1.14 (2019-04-30)

Fixed

• preserve newlines at end of file

• CVE-2019-10906, module now requires jinja2>=2.10.1

0.1.13 (2019-03-29)

Fixed

• Fix for ignored raw tag with two-pass rendering

0.1.12 (2019-03-14)

Added

• add support for two pass rendering

0.1.11 (2019-01-30)

Changed

• Changed behavior of nested keys with the consul tag.

example: in consul we have an key key1/key2/123 envvar KEY=consul:key1/key2 => {{ KEY }} will now render to 123 and no longer to {“key2”: 123}

0.1.10 (2018-09-24)

Added

• add file tag, this tag will place the content of the file into the variable

0.1.9 (2018-09-21)

Fixed

• fix import issue on python 2.7

0.1.8 (2018-09-21)

Changed

• handle hashrocket characters in json tag (as produced by hiera)

0.1.7 (2018-09-20)

Added

• Add list tag which will parse a comma separated list

0.1.6 (2018-05-17)

Changed

• Improved error messages

• Handle json decode errors

0.1.5 (2018-05-11)

Added

• add no-color option

0.1.4 (2017-08-21)

0.1.3 (2017-07-11)

0.1.2 (2017-05-17)

Fixed

• additional fix for install issue on python2 (added MANIFEST.in)

0.1.1 (2017-05-17)

0.1.0 (2017-05-16)

0.0.2 (2017-05-16)

0.0.1 (2017-05-01)

Initial release