Sphinx Documentation Build Automation
Project description
=================================
Giza -- Documentation Build Tools
=================================
Giza is a collection of tools built around `Sphinx <>`_, that
coordinates assembling, building, and deploying documentation. Giza
primarily addresses the MongoDB documentation project; however, its
design is sufficiently generic to be able to facilitate the builds of
multiple documentation resources produced at MongoDB.
Features and Goals
------------------
Giza has the following objectives and primary features:
- Facilitate fully-local test builds. Contributors to the
documentation should be able to generate the documentation using the
exact same process used to produce the production version of the
resources.
- Generate content from structured forms into reStructuredText that
Sphinx can publish. In an effort to manage duplicated content, and
facilitate sustainable content reuse, Giza translates structured
content, procedures, including command line interfaces, tables of
contents, and API interfaces.
- Run multiple Sphinx builds concurrently. Practically speaking,
building the documentation requires running Sphinx multiple
times. Building the documentation requires multiple invocations of
Sphinx, to produce:
- multiple versions of the manual (i.e. HTML, ``json``, PDF, ePub,
etc.)
- transitions of the content in different human languages.
- different editions of a text to address different editions of a
single resource. (e.g. a student and instructor version of a
training resource.)
Internally, Sphinx itself is not optimally parallelized, and
it's considerably more efficient to run multiple Sphinx processes in
parallel, particularly for larger resources and as the matrix of
required build artifacts grows.
Installation
------------
Giza is available in PyPi, and all dependencies and the package
handles all dependencies and version management. For most use, you
will want to install Giza using the specification in the
``requirements.txt`` file in the repository and branch you want to
build. Use the following command: ::
pip install -r requirements.txt
While most branches will use the latest version of the software, at
some point in the future, it may only be possible to build some older
branches with a specific version of Giza. Using ``requirements.txt``
ensure that you will always install the correct version of Giza.
At any time, you can install the latest version with the following
``pip`` command: ::
pip install giza
To install the optional ``github`` and ``jira`` integration, use the
following command:
pip install giza [jira,github]
Use
---
Make Interface
~~~~~~~~~~~~~~
Giza is fully integrated into the ``Makefile`` system present in all
MongoDB documentation repositories. Typically you will run builds
using ``make html``, for the html output, or ``make latex`` for the
PDF build. There are two major important targets: ``publish`` that
builds the full production build locally, and ``push`` that is
equivalent to ``publish`` but also uploads all artifacts the resource
to the production web servers.
During the transition to Giza from the legacy system, all Giza
targets have ``giza-`` prefixes, so to use Giza targets you would use
``make giza-html`` and ``make giza-publish``.
Direct Use
~~~~~~~~~~
While most common operations are wrapped in familiar ``make`` targets,
you can run Giza directly from the command line. This section provides
an overview of these operations. See the output of ``-help`` at all
levels for specific syntax.
Sphinx
``````
The following commands will build the ``html`` version of the
resource: ::
giza sphinx --builder html
giza sphinx -b html
Replace ``html`` with any Sphinx builder you wish to use. You can
specify a list, space separated, of builders to build multiple formats
at once. For example: ::
giza sphinx --builder latex dirhtml json html singlehtml epub man
For the MongoDB Manual this is equivalent to the ``make publish``
operation. For projects that have multiple editions, you can specify
the edition as a section option, for example: ::
giza sphinx --builder latex json html dirhtml singlehtml --edition saas hosted
This is the Giza command to build the full MMS documentation, and it
builds ``saas`` and ``hosted`` versions of the manual for 5 sphinx
output formats.
Diagnostics
```````````
You can use the ``giza config`` command to see a rendered version of
the configuration object used during builds. ``config`` allows you to
see how specifying a language or edition will affect the config
object. For example: ::
giza config
giza config --edition saas
giza config --edition hosted
giza config --edition saas --language es
giza config --edition hosted --language fr
giza config -e saas -l es
giza config -e hosted -l fr
Deploying
`````````
There are two targets that deploy built documentation to the
production environment: ``deploy``, which only uploads the
resources; and ``push``, which builds and then deploys the
resources.
Each branch and repository defines its behavior of the deployment in a
*push* config file. These define a number of "push targets" that
describe how to upload the artifacts. When you run a deploy operation, you
specify one or more of these push targets and Giza will deploy the
artifacts specified in the configuration.
``push`` takes arguments that are the combination of the ``sphinx``
command and the ``deploy`` command. Consider the following commands:
giza deploy --target push
giza deploy --target stage
giza deploy --target push-saas
giza deploy --target stage-saas
giza push --deploy push-saas push-hosted --builder latex json html dirhtml singlehtml --edition saas hosted --language es
giza push --deploy push-saas push-hosted --builder latex json html dirhtml singlehtml --edition saas hosted
Add the ``--dry-run`` or ``-d`` option to any ``deploy`` command to
avoid actually uploading artifacts during testing.
Git
```
Giza provides wrappers for several common ``git`` operations. You can
use Giza to apply the patch from a github pull request or from a
single Github commit: ::
giza git am -p <object>
giza git am --patch <object>
Replace ``<object>`` with the ID of a pull request against the
repository that repository. You can apply any object from github, by
passing a full github URI as the ``<object>``.
All ``giza git`` commands support a ``--branch`` argument that allows
them to perform their operation on a different branch. For example: ::
giza git am --patch 1066 --branch v4.2
giza git am -p 1066 -b v4.2
You can also cherry-pick commits from the local repository onto the
current branch: ::
giza git cp --commits a5b8087
giza git cp -c a5b8087
The ``git cp`` command allows you to cherry pick a list of commits,
but is most useful in combination with the ``--branch`` option to
apply commits to other branches, as in the following examples: ::
giza git cp --commits a5b8087 8f9150a 2eb441b
giza git cp -c a5b8087 8f9150a 2eb441b
giza git cp --commits a5b8087 8f9150a 2eb441b --branch v0.2
giza git cp -c a5b8087 8f9150a 2eb441b --branch v0.2
Additional Giza Operations
``````````````````````````
``generate``
''''''''''''
These operations generate content or fetch data used by the build
without generating the full artifacts. Useful for debugging and
testing. In normal operations the ``sphinx`` operations generate
require inputs, and these operations are not needed.
``generate`` provides the following operations to generate content.
- ``api``
- ``assets``
- ``images``
- ``intersphinx``
- ``options``
- ``primer``
- ``steps``
- ``tables``
- ``toc``
``includes``
''''''''''''
The ``includes`` operations introspect the resources' content reuse,
and allow writers to be able to see the dependency relationship
between source files. ``giza includes`` has the following additional
operations:
- ``recursive``: returns a list of all files that also include other
files.
- ``changes``: returns a list of all files in the repository affected
indirectly by uncommitted changes to the current repository. (Requires
`PyGit2 <https://github.com/libgit2/pygit2>`_)
- ``once``: returns a list of all included files that are only used
once.
- ``unused``: returns a list of all included files that are not used
at all.
- ``graph``: return a document that maps include files to the
dependent source files. Includes the ``--filter`` option, which
allows you to specify a prefix of included files to limit the size
or scope of the graph.
``package``
'''''''''''
Giza provides support for creating "packages" of build artifacts that
you can use to deploy a version of the resource produced on a
different system or at a different time. This makes it possible to
safely deploy a sequence of builds in quick succession. The
``package`` command provides the following options:
- ``create``: Given a *push target*, build ha package of the current
build output and the current configuration object.
- ``fetch``: Given a URL, download the package to the
local "build archive." Will refuse to download a package that
already exists in the build archive.
- ``unwind``: Given a path or URL of a package, extract the package to
the "public output" directory used for staging.
- ``deploy``: Given a *push target* and the path or URL of a package,
extract the package and upload those artifacts.
Additional Components
---------------------
Beyond the ``giza`` command, the Giza package includes several
additional utilities that address various needs of the documentation
project:
``scrumpy`` provides reports that help us manage our SCRUM and sprint,
as well as help with Jira triage and backlog management.
``mdbpr`` is a tool that integrates with Github to identify pull
requests that appropriate for merging.
These components are simple but contain additional dependencies and
do *not* share the same root configuration structure with regards to
the main ``giza`` application. Furthermore, their functionality is not
likely to be relevant for a majority of Giza users.
Giza -- Documentation Build Tools
=================================
Giza is a collection of tools built around `Sphinx <>`_, that
coordinates assembling, building, and deploying documentation. Giza
primarily addresses the MongoDB documentation project; however, its
design is sufficiently generic to be able to facilitate the builds of
multiple documentation resources produced at MongoDB.
Features and Goals
------------------
Giza has the following objectives and primary features:
- Facilitate fully-local test builds. Contributors to the
documentation should be able to generate the documentation using the
exact same process used to produce the production version of the
resources.
- Generate content from structured forms into reStructuredText that
Sphinx can publish. In an effort to manage duplicated content, and
facilitate sustainable content reuse, Giza translates structured
content, procedures, including command line interfaces, tables of
contents, and API interfaces.
- Run multiple Sphinx builds concurrently. Practically speaking,
building the documentation requires running Sphinx multiple
times. Building the documentation requires multiple invocations of
Sphinx, to produce:
- multiple versions of the manual (i.e. HTML, ``json``, PDF, ePub,
etc.)
- transitions of the content in different human languages.
- different editions of a text to address different editions of a
single resource. (e.g. a student and instructor version of a
training resource.)
Internally, Sphinx itself is not optimally parallelized, and
it's considerably more efficient to run multiple Sphinx processes in
parallel, particularly for larger resources and as the matrix of
required build artifacts grows.
Installation
------------
Giza is available in PyPi, and all dependencies and the package
handles all dependencies and version management. For most use, you
will want to install Giza using the specification in the
``requirements.txt`` file in the repository and branch you want to
build. Use the following command: ::
pip install -r requirements.txt
While most branches will use the latest version of the software, at
some point in the future, it may only be possible to build some older
branches with a specific version of Giza. Using ``requirements.txt``
ensure that you will always install the correct version of Giza.
At any time, you can install the latest version with the following
``pip`` command: ::
pip install giza
To install the optional ``github`` and ``jira`` integration, use the
following command:
pip install giza [jira,github]
Use
---
Make Interface
~~~~~~~~~~~~~~
Giza is fully integrated into the ``Makefile`` system present in all
MongoDB documentation repositories. Typically you will run builds
using ``make html``, for the html output, or ``make latex`` for the
PDF build. There are two major important targets: ``publish`` that
builds the full production build locally, and ``push`` that is
equivalent to ``publish`` but also uploads all artifacts the resource
to the production web servers.
During the transition to Giza from the legacy system, all Giza
targets have ``giza-`` prefixes, so to use Giza targets you would use
``make giza-html`` and ``make giza-publish``.
Direct Use
~~~~~~~~~~
While most common operations are wrapped in familiar ``make`` targets,
you can run Giza directly from the command line. This section provides
an overview of these operations. See the output of ``-help`` at all
levels for specific syntax.
Sphinx
``````
The following commands will build the ``html`` version of the
resource: ::
giza sphinx --builder html
giza sphinx -b html
Replace ``html`` with any Sphinx builder you wish to use. You can
specify a list, space separated, of builders to build multiple formats
at once. For example: ::
giza sphinx --builder latex dirhtml json html singlehtml epub man
For the MongoDB Manual this is equivalent to the ``make publish``
operation. For projects that have multiple editions, you can specify
the edition as a section option, for example: ::
giza sphinx --builder latex json html dirhtml singlehtml --edition saas hosted
This is the Giza command to build the full MMS documentation, and it
builds ``saas`` and ``hosted`` versions of the manual for 5 sphinx
output formats.
Diagnostics
```````````
You can use the ``giza config`` command to see a rendered version of
the configuration object used during builds. ``config`` allows you to
see how specifying a language or edition will affect the config
object. For example: ::
giza config
giza config --edition saas
giza config --edition hosted
giza config --edition saas --language es
giza config --edition hosted --language fr
giza config -e saas -l es
giza config -e hosted -l fr
Deploying
`````````
There are two targets that deploy built documentation to the
production environment: ``deploy``, which only uploads the
resources; and ``push``, which builds and then deploys the
resources.
Each branch and repository defines its behavior of the deployment in a
*push* config file. These define a number of "push targets" that
describe how to upload the artifacts. When you run a deploy operation, you
specify one or more of these push targets and Giza will deploy the
artifacts specified in the configuration.
``push`` takes arguments that are the combination of the ``sphinx``
command and the ``deploy`` command. Consider the following commands:
giza deploy --target push
giza deploy --target stage
giza deploy --target push-saas
giza deploy --target stage-saas
giza push --deploy push-saas push-hosted --builder latex json html dirhtml singlehtml --edition saas hosted --language es
giza push --deploy push-saas push-hosted --builder latex json html dirhtml singlehtml --edition saas hosted
Add the ``--dry-run`` or ``-d`` option to any ``deploy`` command to
avoid actually uploading artifacts during testing.
Git
```
Giza provides wrappers for several common ``git`` operations. You can
use Giza to apply the patch from a github pull request or from a
single Github commit: ::
giza git am -p <object>
giza git am --patch <object>
Replace ``<object>`` with the ID of a pull request against the
repository that repository. You can apply any object from github, by
passing a full github URI as the ``<object>``.
All ``giza git`` commands support a ``--branch`` argument that allows
them to perform their operation on a different branch. For example: ::
giza git am --patch 1066 --branch v4.2
giza git am -p 1066 -b v4.2
You can also cherry-pick commits from the local repository onto the
current branch: ::
giza git cp --commits a5b8087
giza git cp -c a5b8087
The ``git cp`` command allows you to cherry pick a list of commits,
but is most useful in combination with the ``--branch`` option to
apply commits to other branches, as in the following examples: ::
giza git cp --commits a5b8087 8f9150a 2eb441b
giza git cp -c a5b8087 8f9150a 2eb441b
giza git cp --commits a5b8087 8f9150a 2eb441b --branch v0.2
giza git cp -c a5b8087 8f9150a 2eb441b --branch v0.2
Additional Giza Operations
``````````````````````````
``generate``
''''''''''''
These operations generate content or fetch data used by the build
without generating the full artifacts. Useful for debugging and
testing. In normal operations the ``sphinx`` operations generate
require inputs, and these operations are not needed.
``generate`` provides the following operations to generate content.
- ``api``
- ``assets``
- ``images``
- ``intersphinx``
- ``options``
- ``primer``
- ``steps``
- ``tables``
- ``toc``
``includes``
''''''''''''
The ``includes`` operations introspect the resources' content reuse,
and allow writers to be able to see the dependency relationship
between source files. ``giza includes`` has the following additional
operations:
- ``recursive``: returns a list of all files that also include other
files.
- ``changes``: returns a list of all files in the repository affected
indirectly by uncommitted changes to the current repository. (Requires
`PyGit2 <https://github.com/libgit2/pygit2>`_)
- ``once``: returns a list of all included files that are only used
once.
- ``unused``: returns a list of all included files that are not used
at all.
- ``graph``: return a document that maps include files to the
dependent source files. Includes the ``--filter`` option, which
allows you to specify a prefix of included files to limit the size
or scope of the graph.
``package``
'''''''''''
Giza provides support for creating "packages" of build artifacts that
you can use to deploy a version of the resource produced on a
different system or at a different time. This makes it possible to
safely deploy a sequence of builds in quick succession. The
``package`` command provides the following options:
- ``create``: Given a *push target*, build ha package of the current
build output and the current configuration object.
- ``fetch``: Given a URL, download the package to the
local "build archive." Will refuse to download a package that
already exists in the build archive.
- ``unwind``: Given a path or URL of a package, extract the package to
the "public output" directory used for staging.
- ``deploy``: Given a *push target* and the path or URL of a package,
extract the package and upload those artifacts.
Additional Components
---------------------
Beyond the ``giza`` command, the Giza package includes several
additional utilities that address various needs of the documentation
project:
``scrumpy`` provides reports that help us manage our SCRUM and sprint,
as well as help with Jira triage and backlog management.
``mdbpr`` is a tool that integrates with Github to identify pull
requests that appropriate for merging.
These components are simple but contain additional dependencies and
do *not* share the same root configuration structure with regards to
the main ``giza`` application. Furthermore, their functionality is not
likely to be relevant for a majority of Giza users.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
giza-0.2.3.tar.gz
(85.4 kB
view hashes)
