Python Forum

Full Version: Automating unittest
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Say I have a couple of unit testing files (this is a stripped down example of my actual problem):

list_test.py
import unittest

class SortTest(unittest.TestCase):
	"""Tests of sorting lists."""

	def setUp(self):
		self.list = [1, 5, 2, 4, 3]

	def testAscending(self):
		"""Test an ascending sort of a list."""
		self.list.sort()
		self.assertEqual([1, 2, 3, 4, 5], self.list)

	def testDescending(self):
		"""Test a descending sort of a list."""
		self.list.sort(reverse = True)
		self.assertEqual([5, 4, 3, 2, 1], self.list)

	def testKey(self):
		"""Test a key-based sort of a list."""
		self.list.sort(key = lambda x: (-x) ** x)
		self.assertEqual([5, 3, 1, 2, 4], self.list)

if __name__ == '__main__':
	unittest.main()
string_test.py
import unittest

class StripTest(unittest.TestCase):
	"""Tests of the strip method of strings."""

	def setUp(self):
		self.text = ' words in spaces '.strip()

	def testLeft(self):
		"""Test that stripping is done on the left."""
		self.assertEqual('w', self.text[0])

	def testMiddle(self):
		"""Test that stripping is not done in the middle."""
		self.assertEqual(' ', self.text[5])

	def testRight(self):
		"""Test that stripping is done on the right."""
		self.assertEqual('s', self.text[-1])

if __name__ == '__main__':
	unittest.main()
And I have a _test_all.py file so that I can run them separately or all at once:

import unittest

if __name__ == '__main__':
	test_suite = unittest.defaultTestLoader.discover('.', pattern = '*_test.py')
	unittest.TextTestRunner(verbosity = 1).run(test_suite)
That's all well and good. It works perfectly, as far as I'm concerned. But say I want to test something like the addition operator (+). It's going to be a lot of tests all of the same format: add a to b and see if equals c. That seemed like a good place for automation, right? After beating my head against it for a couple of days, this is the best I could come up with:

import unittest

def add_tests():
    """Make a test class for the addition tests. (unittest.TestCase)"""
    # Create the base class.
    class AddTest(unittest.TestCase):
        """Test of the Addition modifier in Python."""
        pass
    # Create a function to make the test methods.
    def make_add_test(left, right, result, description):
        def testSomething(self):
            self.assertEqual(result, left + right)
        testSomething.__doc__ = 'Test addition of {}.'.format(description)
        return testSomething
    # Define the tests to run.
    tests = [('testMixedFloats', -1.8, 8.1, 6.3, 'a negative float and a postive float'),
        ('testNegativeFloats', -1.8, -8.1, -9.9, 'two negative floats'),
        ('testPostiveFloats', 1.8, 8.1, 9.9, 'two postiive floats'),
        ('testMixedInts', -1, 2, 1, 'a negative integer and a postive integer'),
        ('testNegativeInts', -1, -2, -3, 'two negative integers'),
        ('testPositiveInts', 1, 2, 3, 'two positive integers')]
    # Add the tests to the class.
    for arguments in tests:
        setattr(AddTest, arguments[0], make_add_test(*arguments[1:]))
    return AddTest
AddTest = add_tests()

if __name__ == '__main__':
    unittest.main()
Now this works. Works fine. But I wouldn't say it works beautiful. Works kind of ugly, me thinks. Anyone know of a better way to do this?

I tried writing a function that created a test suite with all of the tests. However, the _test_all.py module does not detect the resulting test suite. I had to create a different test runner, and the output for those tests was separate from the ones auto-detected by unittest.main(). I tried messing with the setUpClass method of TestCase, but is run after a TestCase with multiple *Test methods is split out into individual classes each with a single test. I could not get that to work. I looked around on SO, but couldn't find anything on point.

Some background on the real problem: It is a large project I've been working on since I retired (about 15 months so far). It currently has about 660 tests spread out over 16 files. I am currently working on fleshing out the skeletal unit tests I somewhat randomly wrote while working on the project code, so I expect those numbers to expand over the coming months. Most of the files have multiple TestCase instances, each with multiple tests. The project is meant to run in 2.7 and 3.x, so I was looking for the test code to run likewise. That leaves out using the subTest method that volcano63 found (hey, someone actually looked at the similar threads list!).

tl;dr: I want to programmatically create a bunch of similar test for unittest that can be found by automatic text detection in 2.7 or 3.x. Any better/prettier/more-pythonic ideas than repeatedly creating a method and adding it to a TestCase?
(Oct-28-2018, 08:39 PM)ichabod801 Wrote: [ -> ]Any better/prettier/more-pythonic ideas than repeatedly creating a method and adding it to a TestCase?
Well yes, I think so. How about a Mako template?
from mako.template import Template
import unittest

testlist = [
    ('testMixedFloats', -1.8, 8.1, 6.3, 'a negative float and a postive float'),
    ('testNegativeFloats', -1.8, -8.1, -9.9, 'two negative floats'),
    ('testPostiveFloats', 1.8, 8.1, 9.9, 'two postiive floats'),
    ('testMixedInts', -1, 2, 1, 'a negative integer and a postive integer'),
    ('testNegativeInts', -1, -2, -3, 'two negative integers'),
    ('testPositiveInts', 1, 2, 3, 'two positive integers')]

template = Template("""\
class AddTest(unittest.TestCase):
% for name, left, right, result, description in testlist:
    def ${name}(self):
        '''Test addition of ${description}'''
        self.assertEqual(${result}, ${left} + ${right})
% endfor
""")

classcode = template.render(testlist=testlist)
print(classcode)
exec(classcode)
print(AddTest)
Output:
class AddTest(unittest.TestCase): def testMixedFloats(self): '''Test addition of a negative float and a postive float''' self.assertEqual(6.3, -1.8 + 8.1) def testNegativeFloats(self): '''Test addition of two negative floats''' self.assertEqual(-9.9, -1.8 + -8.1) def testPostiveFloats(self): '''Test addition of two postiive floats''' self.assertEqual(9.9, 1.8 + 8.1) def testMixedInts(self): '''Test addition of a negative integer and a postive integer''' self.assertEqual(1, -1 + 2) def testNegativeInts(self): '''Test addition of two negative integers''' self.assertEqual(-3, -1 + -2) def testPositiveInts(self): '''Test addition of two positive integers''' self.assertEqual(3, 1 + 2) <class '__main__.AddTest'>
Damn, I knew I forgot something. I'm trying to stick to the standard library. That does look nice, though.

Hmmm, looks like there's no 2.7 version of mako.
I have mako both on 2.7 and 3.5 on my computer (from ubuntu repositories).

You can also do it with the standard library, but it is less robust or extensible
body = '\n'.join(
"""\
    def {}(self):
        '''Test addition of {}'''
        self.assertEqual({}, {} + {})\
""".format(name, description, result, left, right)
for name, left, right, result, description in testlist)
classcode = 'class AddTest(unittest.TestCase):\n' + body
exec(classcode)
print(classcode)
print(AddTest)