QCppUnit Manual
This library consists of a framework to define suites of
tests, and a frontend to run them. This is the heart of Extrem Programming
but is not specific to it. Every programmer is compelled to write automated
tests sometimes in its life.
The test framework was not developed by me, but the Qt dialog was.
TestSearchBackend is the name of the class what will hold your tests for the class being tested. In this case, we test the Search Backend related classes. TestSeachBackend must inherit from TestCase.
This class contains two tests which are testCreate() and testBackend(). Here is for example the beginning of the implementation of testBackend() :
checkEquals( x, y ) is the keyword here. It will make the test fail if x is not equal to y. If no checkEquals fails, the test succeeds. Available checking functions are:
When the test fails, it stops and an error message is provided by the test runner, with the name of the test, the file and the line number where it failed, and in case of a checkEquals, with the expected and the actual value: a failed checkEquals( myVar+1, 3 ) gives "myVar+1 is 2 (0x2) instead of expected 3 (0x3)".
It is possible to define the virtual functions setUp() and tearDown(). setUp() will be called before every test of the test suite and tearDown() after every test. The default implementation does nothing. The function permits to setup some variables that will be used in every test. The variables are usually protected members of your TestCase inherited class.
One you have defined your tests, you must build a suite to package them. This is what the function suite() does:
Yes, you need this complicated TestCaller thing, this is because we use C++. All the TestCaller do is memorize a class and a test function to call, with an assiociated name.
The function returning a suite() is static because it doesn't depend on the object TestCase at all. In fact, the suite could be built outside of the TestCase class if you declare your tests as public.
To run your tests, just create a GuiTestRunner and add your test suites to it:
When you run your tests, you will get something like the following screenshots:
The screenshots speaks for themselves but I will explain them nevertheless.
The progress bar is green when all tests pass, and turns red when one test fails. If all your test pass, you finish with a green progress bar, at 100%, else with a red progress bar, at less than 100%.
At the beginning, the runner displays the list of the test suites with their number of tests and successful tests ( "0/3" ). All the test suite name are written in black.
When you run a suite, each test is passed and the number of successful tests is updated. The progress bar progresses and changes color if some test fail or pass. If all test passes, the name of the suite is written in green and the report is ("3/3"). If some test fails, the name of the suite is written in red and detail information about the test and its failure is appended to the
suite.
There are two options available for GuiTestRunner:
Sometimes, you need to give Qt some time to process its signals between your tests. I had to put the following code inside a test to make a signal execute its slot:
I usually define my test suite inside my TestCase class, as an inline function. This has the drawback that when you add a new test to the suite, you must recompile the file actually using the suite (the one that contains the runner). Else, the inline code has not been updated and your test doesn't show up.
The way we use this framework in Extrem Programming is: First code the test using your class and all the member you need, even if those member do not exist yet. Usually, your test won't even compile. Add the necessary member to make the test compile. Then run the test, see it failing. Then code your feature and run the test again. Debug, run the test again until it passes. When the test passes, you usually improve it. The second version of your test will usually test everything more accurately. Again, code until your test pass. Now that your feature is implemented, have a look at the code you have just written. Can it be improved ? Would renaming a variable make the code clearer ? Is there some similar code somewhere, that it woule make sense to refactor commonly ? Could some calculations be put inside another function ? When you have checked all this and possibly improved your code, run your test a last time. It passes ? Make a little feast, relax five minutes, go get a glass of water, go to the bathroom. Then you are ready for the next feature.
If you are not familiar with Extrem Programming, you could be tempted of
writing all the tests at once, and then make them pass one by one. It would be
a mistake because
The test framework was not developed by me, but the Qt dialog was.
1. Building your test suite
Usually you define your suite of test like that:
class TestSearchBackend : public TestCase
{
public:
TestSearchBackend(const char * name )
: TestCase(name)
{}
virtual void setUp();
virtual void tearDown();
static Test *suite ()
{
TestSuite *testSuite = new TestSuite ("Search Backend");
testSuite->addTest( new TestCaller<TestSearchBackend> ("testCreate", &TestSearchBackend::testCreate ));
testSuite->addTest( new TestCaller<TestSearchBackend> ("testBackend", &TestSearchBackend::testBackend ));
return testSuite;
}
protected:
// My tests
void testCreate();
void testBackend();
// internal variable
int _someVariable;
FakeSearcher * _fs;
};
TestSearchBackend is the name of the class what will hold your tests for the class being tested. In this case, we test the Search Backend related classes. TestSeachBackend must inherit from TestCase.
This class contains two tests which are testCreate() and testBackend(). Here is for example the beginning of the implementation of testBackend() :
void TestSearchBackend::testBackend()
{
Query * q = new FakeQuery("1");
checkEquals( q->getValue(), "1" );
Document * d = new FakeDocument(1);
checkEquals( d->getAuthor(), "a1" );
checkEquals( d->getYear(), "y1" );
checkEquals( d->getTitle(), "t1" );
...
checkEquals( x, y ) is the keyword here. It will make the test fail if x is not equal to y. If no checkEquals fails, the test succeeds. Available checking functions are:
- check( x ) : x must be true
- checkEquals( x, 5 ) : x casted to long must be equals to the value 5
- checkEquals( x, 5.0 ) : x casted to double must be equals to the value 5.0
- checkEquals( x, "5" ) : x casted to QString must be equals to the string "5"
When the test fails, it stops and an error message is provided by the test runner, with the name of the test, the file and the line number where it failed, and in case of a checkEquals, with the expected and the actual value: a failed checkEquals( myVar+1, 3 ) gives "myVar+1 is 2 (0x2) instead of expected 3 (0x3)".
It is possible to define the virtual functions setUp() and tearDown(). setUp() will be called before every test of the test suite and tearDown() after every test. The default implementation does nothing. The function permits to setup some variables that will be used in every test. The variables are usually protected members of your TestCase inherited class.
One you have defined your tests, you must build a suite to package them. This is what the function suite() does:
testSuite->addTest( new TestCaller<TestSearchBackend> ("testCreate", &TestSearchBackend::testCreate));
Yes, you need this complicated TestCaller thing, this is because we use C++. All the TestCaller do is memorize a class and a test function to call, with an assiociated name.
The function returning a suite() is static because it doesn't depend on the object TestCase at all. In fact, the suite could be built outside of the TestCase class if you declare your tests as public.
2. Running the tests
To run the tests, you can use either the Text runner of the Gui runner. Since I focus on Qt, I will only present the Qt runner but the text runner works exactely the same way.To run your tests, just create a GuiTestRunner and add your test suites to it:
The runner will iterate over each suite. For each suite, it will run all the tests (calling setUp() before them and tearDown() after them). The runner displays information for every test that fails: suite name, test name, file, line, error message. When all test have been run, the runner gives a report: "4 of 9 tests successful. 5 tests with failures".
GuiTestRunner * runner = new GuiTestRunner;
Test * suite;
suite = TestWithSuccess::suite();
runner->addTest( suite->toString(), suite );
runner->show();
When you run your tests, you will get something like the following screenshots:
The screenshots speaks for themselves but I will explain them nevertheless.
The progress bar is green when all tests pass, and turns red when one test fails. If all your test pass, you finish with a green progress bar, at 100%, else with a red progress bar, at less than 100%.
At the beginning, the runner displays the list of the test suites with their number of tests and successful tests ( "0/3" ). All the test suite name are written in black.
When you run a suite, each test is passed and the number of successful tests is updated. The progress bar progresses and changes color if some test fail or pass. If all test passes, the name of the suite is written in green and the report is ("3/3"). If some test fails, the name of the suite is written in red and detail information about the test and its failure is appended to the
suite.
There are two options available for GuiTestRunner:
- runTestOnShowing is passed in the constructor, defaults to true. It will make all the test to be run 1 second after the dialog has showed up. If you want to run your tests manually, simply set this argument to false.
- setDisplayDebug can be set to true or false. When set to true, it will print on the stderr (using qDebug) the name of the suite and the test being run. This is very useful when your tests produce some debug output and you want to know which test produce which output. I use it all the time but you can disable it.
3. Examples
I provide two examples of use of this library. Example 1 shows a basic usage of QCppUnit for some non graphical test. Example 2 is more interesting because it tests some user interaction with some Qt widgets. I have written a detailed explanation of how I have built it. Check the html file in the example2 directory.4. Notes
Qt makes it possible to simulate the user's behaviour. You can use for example the function animateClick() on a QPushButton to simulate the user clicking on it. In the very unprobable case where your Qt class doesn't provide any direct way to simulate user behaviour, you can pass QMouseEvent and QKeyEvent to the application loop.Sometimes, you need to give Qt some time to process its signals between your tests. I had to put the following code inside a test to make a signal execute its slot:
If this doesn't work, you can use QTimer with a delay of 0 to ensure all your signal/slots have been processed. This means that your test must be spread accross more than one function, one of them being a slot. But this way, you are sure that the Qt event loop has all the time it needs.
QTime t;
t.start();
while( t.elapsed() < 1000 ) {
qApp->processEvents();
}
I usually define my test suite inside my TestCase class, as an inline function. This has the drawback that when you add a new test to the suite, you must recompile the file actually using the suite (the one that contains the runner). Else, the inline code has not been updated and your test doesn't show up.
The way we use this framework in Extrem Programming is: First code the test using your class and all the member you need, even if those member do not exist yet. Usually, your test won't even compile. Add the necessary member to make the test compile. Then run the test, see it failing. Then code your feature and run the test again. Debug, run the test again until it passes. When the test passes, you usually improve it. The second version of your test will usually test everything more accurately. Again, code until your test pass. Now that your feature is implemented, have a look at the code you have just written. Can it be improved ? Would renaming a variable make the code clearer ? Is there some similar code somewhere, that it woule make sense to refactor commonly ? Could some calculations be put inside another function ? When you have checked all this and possibly improved your code, run your test a last time. It passes ? Make a little feast, relax five minutes, go get a glass of water, go to the bathroom. Then you are ready for the next feature.
If you are not familiar with Extrem Programming, you could be tempted of
writing all the tests at once, and then make them pass one by one. It would be
a mistake because
- it is a tedious task to write many tests without running them while it is fun to write tests and features at the same time and to let one improve the other.
- you are very likely to write unnecessary code, to duplicate things, to forget some things, not to use the correct interface. Briefly: to miss the point. Better get familiar with Extrem Programming : www.xprogramming.org