Detection and Segmentation Accuracy Measures
Project description
Brief Description
DAccuracy (Detection Accuracy) allows to compute
some accuracy measures
on an N-dimensional detection or segmentation image
when the ground-truth is available as a CSV file, an image, or a Numpy file.
It works in 3 contexts:
one-to-one: single ground-truth, single detection image;
one-to-many: unique ground-truth, several detection images (typically obtained by various methods);
many-to-many: set of “(ground-truth, detection image)” pairs.
Example console output (accuracy measures can also be written to a CSV file):
Ground truth = ground-truth.csv Detection = detection.png N ground truths = 55 N detections = 47 True_positive = 43 False_positive = 4 False_negative = 12 Precision = 0.9148936170212766 Recall = 0.7818181818181819 F1_score = 0.8431372549019609 Froc_sample = (4, 0.7818181818181819) Check_tp_fn_equal_gt = 55 Check_tp_fp_equal_dn = 47
Installation
The DAccuracy project is published on the Python Package Index (PyPI) at: https://pypi.org/project/daccuracy. It requires version 3.8, or newer, of the interpreter. It should be installable from Python distribution platforms or Integrated Development Environments (IDEs). Otherwise, it can be installed from a command-line console:
- For all users, after acquiring administrative rights:
First installation: pip install daccuracy
Installation update: pip install --upgrade daccuracy
- For the current user (no administrative rights required):
First installation: pip install --user daccuracy
Installation update: pip install --user --upgrade daccuracy
Documentation
After installation, the daccuracy command should be available from a command-line console. The usage help is obtained with daccuracy --help (see output below).
The ground-truth can be specified through a CSV file, a labeled image, or a labeled Numpy array. The detection can be specified through a labeled image or a labeled Numpy array. A labeled image or Numpy array must have the background labeled with zero, with the objects labeled consecutively from 1.
In CSV format, the ground truth must be specified as one row per object where two columns (typically the first two ones) correspond to the row and column of the object center. Note that “row” and “column” are misleading terms since they can have non-integer values.
Example CSV:
1.2, 2.3 3.4, 4.5
This specifies two ground-truth object centers, the first one being at row 1.2 and column 2.3. Alternatively, the center coordinates can be passed in x/y coordinate system. See the usage help below for details.
The following accuracy measures are computed:
Number of ground-truth objects
Number of detected objects
Number of true positives, false positives, and false negatives
Precision, recall, and F1 score
Free-response Receiver Operating Characteristic (FROC) curve sample: named froc_sample and corresponding to the tuple (false positives, true positive rate)
Values for measure correctness checking: check_tp_fn_equal_gt (true_positives + false_negatives ?=? ground-truths) and check_tp_fp_equal_dn (true_positives + false_positives ?=? detections)
Additionally, if the ground-truth has been passed as an image or a Numpy array, the mean, standard deviation, minimum, and maximum of the following measures are also computed:
Ground-truth/detection overlap (as a percentage with respect to the smaller region among ground-truth and detection)
Ground-truth/detection Jaccard index
Pixel-wise precision, recall, and F1 score
Usage Help:
usage: daccuracy [-h] --gt ground_truth --dn detection [--shifts Dn_shift Dn_shift] [-e] [-t TOLERANCE] [-f {csv,nev}] [-o Output file] [-s] [--no-usage-notice] 3 modes: - one-to-one: one ground-truth (csv, image, or Numpy array) vs. one detection (image or Numpy array) - one-to-many: one ground-truth vs. several detections (folder of detections) - many-to-many: several ground-truths (folder of ground-truths) vs. corresponding detections (folder of detections) In many-to-many mode, each detection file must have a counterpart ground-truth file with the same name, but not necessarily the same extension. With 8-bit image formats, ground-truth and detection cannot contain more than 255 objects. If they do, they could be saved using higher-depth formats. However, it is recommended to save them in NPY or NPZ Numpy formats instead. optional arguments: -h, --help show this help message and exit --gt ground_truth Ground-truth CSV file of centers or labeled image or labeled Numpy array, or ground-truth folder; If CSV, --rAcB (or --xAyB) can be passed additionally to indicate that columns A and B contain the centers' rows and cols, respectively (or x's and y's in x/y mode).Columns must be specified as (possibly sequences of) uppercase letters, as is usual in spreadsheet applications. --dn detection Detection labeled image or labeled Numpy array, or detection folder. --shifts Dn_shift Dn_shift Vertical (row) and horizontal (col) shifts to apply to detection. -e, --exclude-border If present, this option instructs to discard objects touching image border, both in ground-truth and detection. -t TOLERANCE, --tol TOLERANCE, --tolerance TOLERANCE Max ground-truth-to-detection distance to count as a hit (meant to be used when ground-truth is a CSV file of centers). -f {csv,nev}, --format {csv,nev} nev: one "Name = Value"-row per measure; csv: one CSV-row per ground-truth/detection pairs. -o Output file Name-Value or CSV file to store the computed measures, or "-" for console output. -s, --show-image If present, this option instructs to show an image superimposing ground-truth onto detection. --no-usage-notice Silences usage notice about maximum number of objects.
Thanks
The project is developed with PyCharm Community.
The development relies on several open-source packages (see install_requires in setup.py).
The code is formatted by Black, The Uncompromising Code Formatter.
The imports are ordered by isort… your imports, so you don’t have to.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for daccuracy-2021.10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d259ffc7ed11b8e4d966fc7f363c3be3e61127f6a0d944d48bf7812f3c045266 |
|
MD5 | 1c4947eeabc1baf330ea268e0936c7d8 |
|
BLAKE2b-256 | 06e38e572ab3832f01e471f5f0b5070b857ddf665eab3c7624b173d1d2731413 |