Skip to main content

Multi-dimensional parameterized tests for Pytest and Nose

Project description

https://travis-ci.org/akaihola/testdimensions.svg?branch=master

There are multiple ways to write parameterized tests in Python. Unittest has some support these days, Nose allows yielding test cases, Pytest has built-in parameterization support, and the excellent nose_parameterized package enhances these capabilities in most test frameworks.

testdimensions provides a convenient way to write multi-dimensional test matrices in some simple scenarios. If your function accepts multiple arguments and you want to test a cross product set of parameter combinations, testdimensions is for you.

Specify your tests as a table whose:

  • Y axis labels are values for the third-last parameter

  • X axis labels are values for the second-last paremeter

  • cell values are the expected values (last parameter)

  • columns are separated by two spaces (make sure this is true on all rows)

# test_math.py
@pytest_mark_dimensions('base,exponent,expected', '''
    # y: base
    # x: exponent
    # cell: expected

          2    3    9
     0    0    0    0
     1    1    1    1
     2    4    8  512
''')
def test_pow(base, exponent, expected):
    assert math.pow(base, exponent) == expected


@pytest_mark_dimensions('input,function,expected', '''
                round  math.floor  math.ceil
    -1.5         -2.0        -2.0       -1.0
     1.0          1.0         1.0        1.0
     1.6          2.0         1.0        2.0
''')
def test_round_floor_ceil(input, function, expected):
    assert function(input) == expected

Output:

$ pytest -v
=========================== test session starts ===============================
platform linux -- Python 3.5.2, pytest-3.0.3, py-1.4.31, pluggy-0.4.0
collected 18 items

test_math.py::test_pow[0-2-0] PASSED
test_math.py::test_pow[0-3-0] PASSED
test_math.py::test_pow[0-9-0] PASSED
test_math.py::test_pow[1-2-1] PASSED
test_math.py::test_pow[1-3-1] PASSED
test_math.py::test_pow[1-9-1] PASSED
test_math.py::test_pow[2-2-4] PASSED
test_math.py::test_pow[2-3-8] PASSED
test_math.py::test_pow[2-9-512] PASSED
test_math.py::test_round_floor_ceil[-1.5-function0--2.0] PASSED
test_math.py::test_round_floor_ceil[-1.5-function1--2.0] PASSED
test_math.py::test_round_floor_ceil[-1.5-function2--1.0] PASSED
test_math.py::test_round_floor_ceil[1.0-function3-1.0] PASSED
test_math.py::test_round_floor_ceil[1.0-function4-1.0] PASSED
test_math.py::test_round_floor_ceil[1.0-function5-1.0] PASSED
test_math.py::test_round_floor_ceil[1.6-function6-2.0] PASSED
test_math.py::test_round_floor_ceil[1.6-function7-1.0] PASSED
test_math.py::test_round_floor_ceil[1.6-function8-2.0] PASSED

============================ 18 passed in 0.03 seconds ========================

Installation

$ pip install testdimensions

Compatibility

Py2.6

Py2.7

Py3.3

Py3.4

Py3.5

PyPy

nose

no

yes

no

no

yes

no

nose2

no

no

no

no

no

no

py.test

not tested

yes

not tested

not tested

yes

not tested

unittest

no

no

no

no

no

no

unittest2

no

no

no

no

no

no

Dependencies

Exhaustive Usage Examples

The @pytest_mark_dimensions decorator is an extension of @pytest.mark.parametrize and requires a comma-separated list of test parameters as its first argument. The second argument is a multi-line string which defines the actual tests. You can also inject values into the test globals namespace using keyword arguments.

To create higher than two-dimensional tests, just define multiple tables interspersed with values for the additional parameters.

@pytest_mark_dimensions('a,b,expected', '''
            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010
    ''',
    million=1000000)
def test_add(a, b, expected):
    assert a + b == expected


@pytest_mark_dimensions('operation,a,b,expected', '''
    operation = operator.sub

            -10   0    9   million
    -9        1  -9  -18  -1000009
     0       10   0   -9  -million
    10       20  10    1   -999990

    operation = operator.add

            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010

    operation = operator.mul

            -10   0    9   million
    -9       90   0  -81  -9000000
     0        0   0    0         0
    10     -100   0   90  10000000

    ''',
    million=1000000)
def test_arithmetic_operations(operation, a, b, expected):
    assert operation(a, b) == expected

For Nose support, you need to install nose_parameterized and use the @nosedimensions decorator:

@nosedimensions('a,b,expected', '''
            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010
    ''',
    million=1000000)
def test_add(a, b, expected):
    assert a + b == expected


@nosedimensions('operation,a,b,expected', '''
    operation = operator.sub

            -10   0    9   million
    -9        1  -9  -18  -1000009
     0       10   0   -9  -million
    10       20  10    1   -999990

    operation = operator.add

            -10   0   9  million
    -9      -19  -9   0   999991
     0      -10   0   9  million
    10        0  10  19  1000010

    operation = operator.mul

            -10   0    9   million
    -9       90   0  -81  -9000000
     0        0   0    0         0
    10     -100   0   90  10000000

    ''',
    million=1000000)
def test_arithmetic_operations(operation, a, b, expected):
    assert operation(a, b) == expected

Note that you still need to enumerate the test parameters just like with Pytest.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

testdimensions-0.0.1.tar.gz (4.7 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page