runstats 2.0.0

Features

• Pure-Python

• Fully Documented

• 100% Test Coverage

• Numerically Stable

• Optional Cython-optimized Extension (5-100 times faster)

• Statistics summary computes mean, variance, standard deviation, skewness, kurtosis, minimum and maximum.

• Regression summary computes slope, intercept and correlation.

• Developed on Python 3.9

• Tested on CPython 3.6, 3.7, 3.8, 3.9

• Tested on Linux, Mac OS X, and Windows

• Tested using GitHub Actions

Quickstart

Installing RunStats is simple with pip:

$ pip install runstats

You can access documentation in the interpreter with Python’s built-in help function:

>>> import runstats
>>> help(runstats)                             # doctest: +SKIP
>>> help(runstats.Statistics)                  # doctest: +SKIP
>>> help(runstats.Regression)                  # doctest: +SKIP
>>> help(runstats.ExponentialStatistics)       # doctest: +SKIP

Tutorial

The Python RunStats module provides three types for computing running statistics: Statistics, ExponentialStatistics and Regression.The Regression object leverages Statistics internally for its calculations. Each can be initialized without arguments:

>>> from runstats import Statistics, Regression, ExponentialStatistics
>>> stats = Statistics()
>>> regr = Regression()
>>> exp_stats = ExponentialStatistics()

Statistics objects support four methods for modification. Use push to add values to the summary, clear to reset the summary, sum to combine Statistics summaries and multiply to weight summary Statistics by a scalar.

>>> for num in range(10):
...     stats.push(float(num))
>>> stats.mean()
4.5
>>> stats.maximum()
9.0
>>> stats += stats
>>> stats.mean()
4.5
>>> stats.variance()
8.68421052631579
>>> len(stats)
20
>>> stats *= 2
>>> len(stats)
40
>>> stats.clear()
>>> len(stats)
0
>>> stats.minimum()
nan

Use the Python built-in len for the number of pushed values. Unfortunately the Python min and max built-ins may not be used for the minimum and maximum as sequences are expected instead. Therefore, there are minimum and maximum methods provided for that purpose:

>>> import random
>>> random.seed(0)
>>> for __ in range(1000):
...     stats.push(random.random())
>>> len(stats)
1000
>>> min(stats)
Traceback (most recent call last):
    ...
TypeError: ...
>>> stats.minimum()
0.00024069652516689466
>>> stats.maximum()
0.9996851255769114

Statistics summaries provide five measures of a series: mean, variance, standard deviation, skewness and kurtosis:

>>> stats = Statistics([1, 2, 5, 12, 5, 2, 1])
>>> stats.mean()
4.0
>>> stats.variance()
15.33333333333333
>>> stats.stddev()
3.915780041490243
>>> stats.skewness()
1.33122127314735
>>> stats.kurtosis()
0.5496219281663506

All internal calculations use Python’s float type.

Like Statistics, the Regression type supports some methods for modification: push, clear and sum:

>>> regr.clear()
>>> len(regr)
0
>>> for num in range(10):
...     regr.push(num, num + 5)
>>> len(regr)
10
>>> regr.slope()
1.0
>>> more = Regression((num, num + 5) for num in range(10, 20))
>>> total = regr + more
>>> len(total)
20
>>> total.slope()
1.0
>>> total.intercept()
5.0
>>> total.correlation()
1.0

Regression summaries provide three measures of a series of pairs: slope, intercept and correlation. Note that, as a regression, the points need not exactly lie on a line:

>>> regr = Regression([(1.2, 1.9), (3, 5.1), (4.9, 8.1), (7, 11)])
>>> regr.slope()
1.5668320150154176
>>> regr.intercept()
0.21850113956294415
>>> regr.correlation()
0.9983810791694997

Both constructors accept an optional iterable that is consumed and pushed into the summary. Note that you may pass a generator as an iterable and the generator will be entirely consumed.

The ExponentialStatistics are constructed by providing a decay rate, initial mean, and initial variance. The decay rate has default 0.9 and must be between 0 and 1. The initial mean and variance default to zero.

>>> exp_stats = ExponentialStatistics()
>>> exp_stats.decay
0.9
>>> exp_stats.mean()
0.0
>>> exp_stats.variance()
0.0

The decay rate is the weight by which the current statistics are discounted by. Consequently, (1 - decay) is the weight of the new value. Like the Statistics class, there are four methods for modification: push, clear, sum and multiply.

>>> for num in range(10):
...     exp_stats.push(num)
>>> exp_stats.mean()
3.486784400999999
>>> exp_stats.variance()
11.593430921943071
>>> exp_stats.stddev()
3.4049127627507683

The decay of the exponential statistics can also be changed. The value must be between 0 and 1.

>>> exp_stats.decay
0.9
>>> exp_stats.decay = 0.5
>>> exp_stats.decay
0.5
>>> exp_stats.decay = 10
Traceback (most recent call last):
  ...
ValueError: decay must be between 0 and 1

The clear method allows to optionally set a new mean, new variance and new decay. If none are provided mean and variance reset to zero, while the decay is not changed.

>>> exp_stats.clear()
>>> exp_stats.decay
0.5
>>> exp_stats.mean()
0.0
>>> exp_stats.variance()
0.0

Combining ExponentialStatistics is done by adding them together. The mean and variance are simply added to create a new object. To weight each ExponentialStatistics, multiply them by a constant factor. If two ExponentialStatistics are added then the leftmost decay is used for the new object. The len method is not supported.

>>> alpha_stats = ExponentialStatistics(iterable=range(10))
>>> beta_stats = ExponentialStatistics(decay=0.1)
>>> for num in range(10):
...     beta_stats.push(num)
>>> exp_stats = beta_stats * 0.5 + alpha_stats * 0.5
>>> exp_stats.decay
0.1
>>> exp_stats.mean()
6.187836645

All internal calculations of the Statistics and Regression classes are based entirely on the C++ code by John Cook as posted in a couple of articles:

• Computing Skewness and Kurtosis in One Pass

• Computing Linear Regression in One Pass

The ExponentialStatistics implementation is based on:

• Finch, 2009, Incremental Calculation of Weighted Mean and Variance

The pure-Python version of RunStats is directly available if preferred.

>>> import runstats.core   # Pure-Python
>>> runstats.core.Statistics
<class 'runstats.core.Statistics'>

When importing from runstats the Cython-optimized version _core is preferred and the core version is used as fallback. Micro-benchmarking Statistics and Regression by calling push repeatedly shows the Cython-optimized extension as 20-40 times faster than the pure-Python extension.

Reference and Indices

• RunStats Documentation

• RunStats API Reference

• RunStats at PyPI

• RunStats at GitHub

• RunStats Issue Tracker

License

Copyright 2013-2021 Grant Jenks

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.