pymultigen 0.1.1

Using the Jinja2 integration

You may want to check out pyecoregen, a code generator from pyecore-based models to Python classes. It is a concrete Jinja2-based code generator built with pymultigen.

Jinja2 is a template-based text generator. Writing a file-generator with Jinja therefore involves writing a template for each type of output file. In pymultigen you will then implement a Task class per output file type, i.e. per Jinja template.

The general form of such a Task looks like this:

class MyTask(multigen.jinja.JinjaTask):

    # Name of template file used by this task.
    template_name = 'name-of-template.tpl'

    def filtered_elements(self, model):
        """Return iterator based over elements in model that are passed to template."""

    def relative_path_for_element(self, element):
        """Returns relative file path receiving the generator output for given element."""

The workflow engine will initially call filtered_elements. This method is expected to return an interator over model elements for which a single file needs to be generated. Model is meant here in an abstract way: It may be an instance of a formal metamodel, but it could be any Python object, like a dictionaries or lists. The contained elements being iterated over are accessible from within a template as element.

Once Jinja has produced a textual result it must be written to file. This is where relative_path_for_element comes into play. For a given element that was filtered from the model before, it returns the corresponding filepath. Note that this path is interpreted to be relative to the top-level output path of the overall generation (see below). If subfolders are mentioned here, they are created on demand.

One of more tasks classes like this must then be registered with a top-level generator. Just like before, a new Generator class is derived from the appropriate base class:

class MyGenerator(multigen.jinja.JinjaGenerator):

    # List of task objects to be processed by this generator.
    tasks = [
        MyTask(),
    ]

    # Root path where Jinja templates are found.
    templates_path = os.path.join(
        os.path.abspath(os.path.dirname(__file__)),
        'templates'
    )

    def create_environment(self, **kwargs):
        """Create Jinja2 environment."""
        environment = super().create_environment(**kwargs)
        # Do any customization of environment here, or delete this method.
        return environment

The base class implementation of {{create_environment}} passes {{templates_path}} to the created environment object to allow Jinja to find the template names specified in a Tasks’s template_name. By overriding this method you can extend the environment, e.g. to add filters and tests. Of course you can also completely replace the implementation, e.g. to change the way how templates how looked up.

The example above simply instantiates the new Task class. Here you can optionally pass a formatter function, that is then applied to the output of Jinja. Formatters are simple string transformations, some of which are built-in in the formatters.py module. If you actually are writing a Python code generator you may want to clean up the generated code according to pep8, simply pass the appropriate formatter during task instantiation:

class MyGeneratorWithPep8(multigen.jinja.JinjaGenerator):

    # List of task objects to be processed by this generator.
    tasks = [
        MyTask(formatter=multigen.formatter.format_autopep8),
    ]

    ...

Formatters

Writing a new formatter is trivial: Simply create a function that transforms an input string into the nicely formatted output string. If you want to get your formatter added to pymultigen, please make sure that:

• New dependencies (like autopep8 in the existing pep8 formatter) are only imported in the formatting function. This way user only pay for what they use.

• Please write unittests and add your possible dependencies to the tests_require argument in setup.py.

There is not much more to it.

Templating engine

For a live sample, look at the Jinja2 integration in jinja.py. For your templating engine X, you probably have to write small Generator and Task base classes like this:

class XGenerator(TemplateGenerator):

    def __init__(self, environment=None, **kwargs):
        super().__init__(**kwargs)
        # Add any attributes to the generator that are static with respect to a full generation
        # run (over all files), like a Jinja2 environment.
        ...


class XTask(TemplateFileTask):

    def generate_file(self, element, filepath):
        """Actual generation of element."""

Each element that is iterated over from the input model is eventually passed to the tasks’s generate_file method. Here simply call you template engine to produce the output string. You also want to apply the optional formatter before writing the string to disk. This is how the Jinja task does it:

def generate_file(self, element, filepath):
    template = self.environment.get_template(self.template_name)
    context = self.create_template_context(element=element)

    with open(filepath, 'wt') as file:
        file.write(self.formatter(template.render(**context)))

The implementation shows two more things:

• The template to be used is retrieved from an environment that is specific to the template engine. Such an environment is usually passed down from the Generator class to the Task.

• create_template_context is a function implemented in base class TemplateTask. It implements the very common case of dictionaries being used as template context objects. Of course you can override this if it doesn’t match your engine.