daccuracy 2022.18

3.1- Input Formats

The ground-truth can be specified through a CSV file, a labeled image, or a labeled Numpy array. The detection can be specified through a labeled image or a labeled Numpy array. A labeled image or Numpy array must have the background labeled with zero, with the objects labeled consecutively from 1.

In CSV format, the ground-truth must be specified as one row per object where n columns (the first n ones by default) correspond to the row, column, and remaining n-2 coordinates of the object center. Note that these coordinates can have floating-point values (as opposed to being restricted to integers). See the usage help below for details.

Example CSV:

1.2, 2.3
3.4, 4.5

This specifies two ground-truth object centers in dimension 2, the first one being at row 1.2 and column 2.3. Alternatively, the center coordinates can be passed in x/y coordinate system. See the usage help below for details.

3.2- Accuracy Measures

The following accuracy measures are computed:

• Number of ground-truth objects

• Number of detected objects

• Number of true positives, false positives, and false negatives

• Precision, recall, and F1 score

• Free-response Receiver Operating Characteristic (FROC) curve sample: named froc_sample and corresponding to the tuple (false positives, true positive rate)

• Values for measure correctness checking: check_tp_fn_equal_gt (true_positives + false_negatives ?=? ground-truths) and check_tp_fp_equal_dn (true_positives + false_positives ?=? detections)

Additionally, if the ground-truth has been passed as an image or a Numpy array, the mean, standard deviation, minimum, and maximum of the following measures are also computed:

• Ground-truth/detection overlap (as a percentage with respect to the smaller region among ground-truth and detection)

• Ground-truth/detection Jaccard index

• Pixel-wise precision, recall, and F1 score

3.3- Output Formats

See usage help below.

3.4- Usage Help (daccuracy --help)

Usage Help:

usage: daccuracy [-h] --gt ground_truth --dn detection [--shifts Dn_shift Dn_shift] [-e] [-t TOLERANCE] [-f {csv,nev}]
                 [-o Output file] [-s]

3 modes:
    - one-to-one: one ground-truth (csv, image, or Numpy array) vs. one detection (image or Numpy array)
    - one-to-many: one ground-truth vs. several detections (folder of detections)
    - many-to-many: several ground-truths (folder of ground-truths) vs. corresponding detections (folder of detections)

In many-to-many mode, each detection file must have a counterpart ground-truth file with the same name, but not
necessarily the same extension.

With 8-bit image formats, ground-truth and detection cannot contain more than 255 objects. If they do, they could be
saved using higher-depth formats. However, it is recommended to save them in NPY or NPZ Numpy formats instead.

optional arguments:
  -h, --help            show this help message and exit
  --gt ground_truth     Ground-truth CSV file of centers or labeled image or labeled Numpy array, or ground-truth folder;
                        If CSV, --rAcB (or --xAyB) can be passed additionally to indicate that columns A and B contain
                        the centers' rows and cols, respectively (or x's and y's in x/y mode). Columns must be specified
                        as (possibly sequences of) uppercase letters, as is usual in spreadsheet applications. For
                        ground-truths of dimension "n" higher than 2, the symbol "+" must be used for the remaining
                        "n-2" dimensions. For example, --rAcB+C+D in dimension 4.
  --relabel-gt {seq,full}
                        If present, this option instructs to relabel the ground-truth
                        sequentially.
  --dn detection        Detection labeled image or labeled Numpy array, or detection folder.
  --relabel-gt {seq,full}
                        If present, this option instructs to relabel the ground-truth
                        sequentially.
  --shifts Dn_shift [Dn_shift ...]
                        Vertical (row), horizontal (col), and higher dimension shifts to apply to detection. Default:
                        all zeroes.
  -e, --exclude-border  If present, this option instructs to discard objects touching image border, both in ground-truth
                        and detection.
  -t TOLERANCE, --tol TOLERANCE, --tolerance TOLERANCE
                        Max ground-truth-to-detection distance to count as a hit (meant to be used when ground-truth is
                        a CSV file of centers). Default: zero.
  -f {csv,nev}, --format {csv,nev}
                        nev: one "Name = Value"-row per measure; csv: one CSV-row per ground-truth/detection pairs.
                        Default: "nev".
  -o Output file        CSV file to store the computed measures or "-" for console output. Default: console output.
  -s, --show-image      If present, this option instructs to show an image superimposing ground-truth onto detection.
                        It is actually done only for 2-dimensional images.