spacy-loggers: Logging utilities for spaCy

Starting with spaCy v3.2, alternate loggers are moved into a separate package so that they can be added and updated independently from the core spaCy library.

spacy-loggers currently provides loggers for:

• Weights & Biases
• MLflow

If you'd like to add a new logger or logging option, please submit a PR to this repo!

Setup and installation

spacy-loggers should be installed automatically with spaCy v3.2+, so you usually don't need to install it separately. You can install it with pip or from the conda channel conda-forge:

pip install spacy-loggers

conda install -c conda-forge spacy-loggers

Loggers

WandbLogger

Installation

This logger requires wandb to be installed and configured:

pip install wandb
wandb login

Usage

spacy.WandbLogger.v4 is a logger that sends the results of each training step to the dashboard of the Weights & Biases tool. To use this logger, Weights & Biases should be installed, and you should be logged in. The logger will send the full config file to W&B, as well as various system information such as memory utilization, network traffic, disk IO, GPU statistics, etc. This will also include information such as your hostname and operating system, as well as the location of your Python executable.

Note that by default, the full (interpolated) training config is sent over to the W&B dashboard. If you prefer to exclude certain information such as path names, you can list those fields in "dot notation" in the remove_config_values parameter. These fields will then be removed from the config before uploading, but will otherwise remain in the config file stored on your local system.

Example config

[training.logger]
@loggers = "spacy.WandbLogger.v4"
project_name = "monitor_spacy_training"
remove_config_values = ["paths.train", "paths.dev", "corpora.train.path", "corpora.dev.path"]
log_dataset_dir = "corpus"
model_log_interval = 1000

MLflowLogger

Installation

This logger requires mlflow to be installed and configured:

pip install mlflow

Usage

spacy.MLflowLogger.v1 is a logger that tracks the results of each training step using the MLflow tool. To use this logger, MLflow should be installed. At the beginning of each model training operation, the logger will initialize a new MLflow run and set it as the active run under which metrics and parameters wil be logged. The logger will then log the entire config file as parameters of the active run. After each training step, the following actions are performed:

• The final score is logged under the metric score.
• Individual component scores are logged under their default names.
• Loss values of different components are logged with the loss_ prefix.
• If the final score is higher than the previous best score (for the current run), the model artifact is additionally uploaded to MLflow. This action is only performed if the output_path argument is provided during the training pipeline initialization phase.

By default, the tracking API writes data into files in a local ./mlruns directory.

Note that by default, the full (interpolated) training config is sent over to MLflow. If you prefer to exclude certain information such as path names, you can list those fields in "dot notation" in the remove_config_values parameter. These fields will then be removed from the config before uploading, but will otherwise remain in the config file stored on your local system.

Example config

[training.logger]
@loggers = "spacy.MLflowLogger.v1"
experiment_id = "1"
run_name = "with_fast_alignments"
nested = False
remove_config_values = ["paths.train", "paths.dev", "corpora.train.path", "corpora.dev.path"]