skip to navigation
skip to content

Not Logged In

UnittestRandGenState 0.1.4

Smart random-generation state persistence for unittest

Introduction

Randomized unit tests are great. The inputs fed to your algorithm are not consistent across executions, and so each time the tests run successfully, your confidence in the algorithm grows. Unfortunately, when a test fails, this inconsistency makes it hard to duplicate the input until the problem is solved.

This library provides a simple metaclass for dropping into unittest.TestCase that addresses this. As long as a test fails, it will continue making the same random choices (via random or numpy.random) each execution. Any test that passed the previous execution, will get a fresh set of random choices.

Example

Consider the following simplified example of a test case with three functions testing some algorithms. The first and third algorithms are OK, but the second has a bug when given a number that divides by 20 (the simplified example asserts this directly).

import random


class TestMyAlgs(unittest.TestCase):
    # This "algorithm" is OK.
    def test_ok(self):
        i = random.randint(100, 200)
        self.assertLess(i, i + 1)

    # This "algorithm" fails for inputs that divide by 20.
    def test_bad(self):
        i = random.randint(100, 200)
        self.assertNotEqual(i % 20, 0, i)

    # This "algorithm" is OK too.
    def test_another_ok(self):
        i = random.randint(100, 200)
        self.assertLess(i - 1, i)

The second function will fail approximately 5% of the executions. When it does so, it might be hard (especially in a real-life case) to reproduce the problem, and eventually solve it.

Using this library, a simple change makes things easier. Simply use unittest_rand_gen_state.Saver as the metaclass:

import random

import unittest_rand_gen_state


class TestMyAlgs(unittest.TestCase, metaclass = unittest_rand_gen_state.Saver):
    ...

or, using an older Python version:

import random

import unittest_rand_gen_state


class TestMyAlgs(unittest.TestCase):
    __metaclass__ = unittest_rand_gen_state.Saver
    ...

Nothing else needs to be changed.

Once TestMyAlgs.test_bad first fails in some execution, then it will consistently fail in following executions. That is, say it fails the first time with

======================================================================
FAIL: test_bad (__main__.TestMyAlgs)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/unittest_rand_gen_state/__init__.py", line 75, in wrapped
    fn(*args, **kwargs)
  File "__init__.py", line 21, in test_bad
    self.assertNotEqual(i % 20, 0, i)
AssertionError: 140

----------------------------------------------------------------------

then every following execution will fail exactly the same way, until it is fixed.

The two other tests, meanwhile, will run afresh all the while.

Download, Installation, And Bugtracking

  • The package is at PyPI.

  • The usual setup for Python libraries is used. Type:

    $ pip install UnittestRandGenState

    or

    $ sudo pip install UnittestRandGenState

  • Bugtracking is on Google Code.

Extending

The metaclass supplied by this library deals with the states of random and numpy.random, which should be sufficient for almost all tests. However, the library may be extended by dropping as a metaclass, a subclass of unittest_rand_gen_state.Saver supplying three class methods:

  • reset_state: A method specifying the meaning of resetting the random-generation state (e.g., random.seed)
  • get_state: A method specifying the query to call for getting the current random-generation state (e.g., random.getstate)
  • set_state: A method specifying the function to call to restore the random-generation state using the value returned by get_state (e.g., random.setstate)

E.g., suppose a random-generation library, groovy_rng has the functions groovy_rng.seed (setting the state to some function determined by the current time), groovy_rng.get_state, returning an int, and groovy_rng.set_state taking an int, then here is a metaclass for working with groovy_rng:

class GroovyRNGSaver(unittest_rand_gen_state.Saver):
    reset_state = groovy_rng.seed
    get_state = groovy_rng.get_state
    set_state = groovy_rng.set_state

This metaclass can be used just like in the previous example.

Changes

Changes
Version Date Description
0.1.4. 15/03/2013 Duplicate class-name bugfix, Python 2/3 coexistance bugfix
0.1.3 23/02/2013 Better test hashes, better support for extension
0.1.2 22/02/2013 Doc bugfix
0.1.1 20/02/2013 Less stateful design
0.1.0 18/02/2013 Initial Release
 
File Type Py Version Uploaded on Size
UnittestRandGenState-0.1.4.tar.gz (md5) Source 2013-03-15 6KB
UnittestRandGenState-0.1.4.zip (md5) Source 2013-03-15 12KB
  • Downloads (All Versions):
  • 1 downloads in the last day
  • 105 downloads in the last week
  • 517 downloads in the last month