Skip to main content

A lightweight library to compute Diarization Error Rate (DER).

Project description

SimpleDER Build Status PyPI Version Python Versions Downloads codecov Documentation

Overview

This is a lightweight library to compute Diarization Error Rate (DER).

Features NOT supported:

  • Handling overlapped speech, i.e. two speakers speaking at the same time.
  • Allowing segment boundary tolerance, a.k.a. the collar value.

For more sophisticated metrics with these supports, please use pyannote-metrics instead.

To learn more about speaker diarization, here is a curated list of resources: awesome-diarization.

Diarization Error Rate

Diarization Error Rate (DER) is the most commonly used metrics for speaker diarization.

Its strict form is:

       False Alarm + Miss + Overlap + Confusion
DER = ------------------------------------------
                   Reference Length

The definition of each term:

  • Reference Length: The total length of the reference (ground truth).
  • False Alarm: Length of segments which are considered as speech in hypothesis, but not in reference.
  • Miss: Length of segments which are considered as speech in reference, but not in hypothesis.
  • Overlap: Length of segments which are considered as overlapped speech in hypothesis, but not in reference. This library does NOT support overlap.
  • Confusion: Length of segments which are assigned to different speakers in hypothesis and reference (after applying an optimal assignment).

The unit of each term is seconds.

Note that DER can theoretically be larger than 1.0.

References:

Tutorial

Install

Install the package by:

pip3 install simpleder

or

python3 -m pip install simpleder

API

Here is a minimal example:

import simpleder

# reference (ground truth)
ref = [("A", 0.0, 1.0),
       ("B", 1.0, 1.5),
       ("A", 1.6, 2.1)]

# hypothesis (diarization result from your algorithm)
hyp = [("1", 0.0, 0.8),
       ("2", 0.8, 1.4),
       ("3", 1.5, 1.8),
       ("1", 1.8, 2.0)]

error = simpleder.DER(ref, hyp)

print("DER={:.3f}".format(error))

This should output:

DER=0.350

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

simpleder-0.0.3.tar.gz (4.0 kB view hashes)

Uploaded Source

Built Distribution

simpleder-0.0.3-py3-none-any.whl (8.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page