+ +

+ SimpleTest pretty much follows the MVC pattern + (Model-View-Controller). + The reporter classes are the view and the model is your + test cases and their hiearchy. + The controller is mostly hidden from the user of + SimpleTest unless you want to change how the test cases + are actually run, in which case it is possible to + override the runner objects from within the test case. + As usual with MVC, the controller is mostly undefined + and there are other places to control the test run. +

+ +

Reporting results in HTML

+ The default test display is minimal in the extreme. + It reports success and failure with the conventional red and + green bars and shows a breadcrumb trail of test groups + for every failed assertion. + Here's a fail... +

File test

+ Fail: createnewfile->True assertion failed.
+

1/1 test cases complete. + 0 passes, 1 fails and 0 exceptions.

+ And here all tests passed... +

File test

1/1 test cases complete. + 1 passes, 0 fails and 0 exceptions.

+ The good news is that there are several points in the display + hiearchy for subclassing. +

+ For web page based displays there is the + HtmlReporter class with the following + signature... +

+class HtmlReporter extends SimpleReporter {
+    public HtmlReporter($encoding) { ... }
+    public makeDry(boolean $is_dry) { ... }
+    public void paintHeader(string $test_name) { ... }
+    public void sendNoCacheHeaders() { ... }
+    public void paintFooter(string $test_name) { ... }
+    public void paintGroupStart(string $test_name, integer $size) { ... }
+    public void paintGroupEnd(string $test_name) { ... }
+    public void paintCaseStart(string $test_name) { ... }
+    public void paintCaseEnd(string $test_name) { ... }
+    public void paintMethodStart(string $test_name) { ... }
+    public void paintMethodEnd(string $test_name) { ... }
+    public void paintFail(string $message) { ... }
+    public void paintPass(string $message) { ... }
+    public void paintError(string $message) { ... }
+    public void paintException(string $message) { ... }
+    public void paintMessage(string $message) { ... }
+    public void paintFormattedMessage(string $message) { ... }
+    protected string _getCss() { ... }
+    public array getTestList() { ... }
+    public integer getPassCount() { ... }
+    public integer getFailCount() { ... }
+    public integer getExceptionCount() { ... }
+    public integer getTestCaseCount() { ... }
+    public integer getTestCaseProgress() { ... }
+}
+

+ Here is what some of these methods mean. First the display methods + that you will probably want to override... +

+ HtmlReporter(string $encoding)
+ is the constructor. + Note that the unit test sets up the link to the display + rather than the other way around. + The display is a mostly passive receiver of test events. + This allows easy adaption of the display for other test + systems beside unit tests, such as monitoring servers. + The encoding is the character encoding you wish to + display the test output in. + In order to correctly render debug output when + using the web tester, this should match the encoding + of the site you are trying to test. + The available character set strings are described in + the PHP html_entities() + function. +
+ void paintHeader(string $test_name)
+ is called once at the very start of the test when the first + start event arrives. + The first start event is usually delivered by the top level group + test and so this is where $test_name + comes from. + It paints the page titles, CSS, body tag, etc. + It returns nothing (void). +
+ void paintFooter(string $test_name)
+ Called at the very end of the test to close any tags opened + by the page header. + By default it also displays the red/green bar and the final + count of results. + Actually the end of the test happens when a test end event + comes in with the same name as the one that started it all + at the same level. + The tests nest you see. + Closing the last test finishes the display. +
+ void paintMethodStart(string $test_name)
+ is called at the start of each test method. + The name normally comes from method name. + The other test start events behave the same way except + that the group test one tells the reporter how large + it is in number of held test cases. + This is so that the reporter can display a progress bar + as the runner churns through the test cases. +
+ void paintMethodEnd(string $test_name)
+ backs out of the test started with the same name. +
+ void paintFail(string $message)
+ paints a failure. + By default it just displays the word fail, a breadcrumbs trail + showing the current test nesting and the message issued by + the assertion. +
+ void paintPass(string $message)
+ by default does nothing. +
+ string _getCss()
+ Returns the CSS styles as a string for the page header + method. + Additional styles have to be appended here if you are + not overriding the page header. + You will want to use this method in an overriden page header + if you want to include the original CSS. +

+ There are also some accessors to get information on the current + state of the test suite. + Use these to enrich the display... +

+ array getTestList()
+ is the first convenience method for subclasses. + Lists the current nesting of the tests as a list + of test names. + The first, most deeply nested test, is first in the + list and the current test method will be last. +
+ integer getPassCount()
+ returns the number of passes chalked up so far. + Needed for the display at the end. +
+ integer getFailCount()
+ is likewise the number of fails so far. +
+ integer getExceptionCount()
+ is likewise the number of errors so far. +
+ integer getTestCaseCount()
+ is the total number of test cases in the test run. + This includes the grouping tests themselves. +
+ integer getTestCaseProgress()
+ is the number of test cases completed so far. +

+ One simple modification is to get the HtmlReporter to display + the passes as well as the failures and errors... +

+class ShowPasses extends HtmlReporter {
+    
+    function paintPass($message) {
+        parent::paintPass($message);
+        print "&<span class=\"pass\">Pass</span>: ";
+        $breadcrumb = $this->getTestList();
+        array_shift($breadcrumb);
+        print implode("-&gt;", $breadcrumb);
+        print "-&gt;$message<br />\n";
+    }
+    
+    function _getCss() {
+        return parent::_getCss() . ' .pass { color: green; }';
+    }
+}
+

+ One method that was glossed over was the makeDry() + method. + If you run this method, with no parameters, on the reporter + before the test suite is run no actual test methods + will be called. + You will still get the events of entering and leaving the + test methods and test cases, but no passes or failures etc, + because the test code will not actually be executed. +

+ The reason for this is to allow for more sophistcated + GUI displays that allow the selection of individual test + cases. + In order to build a list of possible tests they need a + report on the test structure for drawing, say a tree view + of the test suite. + With a reporter set to dry run that just sends drawing events + this is easily accomplished. +

+ +

Extending the reporter

+ Rather than simply modifying the existing display, you might want to + produce a whole new HTML look, or even generate text or XML. + Rather than override every method in + HtmlReporter we can take one + step up the class hiearchy to SimpleReporter + in the simple_test.php source file. +

+ A do nothing display, a blank canvas for your own creation, would + be... +

+require_once('simpletest/simple_test.php');
+
+class MyDisplay extends SimpleReporter {
+    
+    function paintHeader($test_name) {
+    }
+    
+    function paintFooter($test_name) {
+    }
+    
+    function paintStart($test_name, $size) {
+        parent::paintStart($test_name, $size);
+    }
+    
+    function paintEnd($test_name, $size) {
+        parent::paintEnd($test_name, $size);
+    }
+    
+    function paintPass($message) {
+        parent::paintPass($message);
+    }
+    
+    function paintFail($message) {
+        parent::paintFail($message);
+    }
+}
+

+ No output would come from this class until you add it. +

+ +

The command line reporter

+ SimpleTest also ships with a minimal command line reporter. + The interface mimics JUnit to some extent, but paints the + failure messages as they arrive. + To use the command line reporter simply substitute it + for the HTML version... +

+<?php
+require_once('simpletest/unit_tester.php');
+require_once('simpletest/reporter.php');
+
+$test = &new TestSuite('File test');
+$test->addTestFile('tests/file_test.php');
+$test->run(new TextReporter());
+?>
+

+ Then invoke the test suite from the command line... +

+php file_test.php
+

+ You will need the command line version of PHP installed + of course. + A passing test suite looks like this... +

+File test
+OK
+Test cases run: 1/1, Failures: 0, Exceptions: 0
+

+ A failure triggers a display like this... +

+File test
+1) True assertion failed.
+    in createnewfile
+FAILURES!!!
+Test cases run: 1/1, Failures: 1, Exceptions: 0
+

+ One of the main reasons for using a command line driven + test suite is of using the tester as part of some automated + process. + To function properly in shell scripts the test script should + return a non-zero exit code on failure. + If a test suite fails the value false + is returned from the SimpleTest::run() + method. + We can use that result to exit the script with the desired return + code... +

+<?php
+require_once('simpletest/unit_tester.php');
+require_once('simpletest/reporter.php');
+
+$test = &new TestSuite('File test');
+$test->addTestFile('tests/file_test.php');
+exit ($test->run(new TextReporter()) ? 0 : 1);
+?>
+

+ Of course we don't really want to create two test scripts, + a command line one and a web browser one, for each test suite. + The command line reporter includes a method to sniff out the + run time environment... +

+<?php
+require_once('simpletest/unit_tester.php');
+require_once('simpletest/reporter.php');
+
+$test = &new TestSuite('File test');
+$test->addTestFile('tests/file_test.php');
+if (TextReporter::inCli()) {
+    exit ($test->run(new TextReporter()) ? 0 : 1);
+}
+$test->run(new HtmlReporter());
+?>
+

+ This is the form used within SimpleTest itself. +

+ +

Remote testing

+ SimpleTest ships with an XmlReporter class + used for internal communication. + When run the output looks like... +

+<?xml version="1.0"?>
+<run>
+  <group size="4">
+    <name>Remote tests</name>
+    <group size="4">
+      <name>Visual test with 48 passes, 48 fails and 4 exceptions</name>
+      <case>
+        <name>testofunittestcaseoutput</name>
+        <test>
+          <name>testofresults</name>
+          <pass>This assertion passed</pass>
+          <fail>This assertion failed</fail>
+        </test>
+        <test>
+          ...
+        </test>
+      </case>
+    </group>
+  </group>
+</run>
+

+ You can make use of this format with the parser + supplied as part of SimpleTest itself. + This is called SimpleTestXmlParser and + resides in xml.php within the SimpleTest package... +

+<?php
+require_once('simpletest/xml.php');
+    
+...
+$parser = &new SimpleTestXmlParser(new HtmlReporter());
+$parser->parse($test_output);
+?>
+

+ The $test_output should be the XML format + from the XML reporter, and could come from say a command + line run of a test case. + The parser sends events to the reporter just like any + other test run. + There are some odd occasions where this is actually useful. +

+ A problem with large test suites is thet they can exhaust + the default 8Mb memory limit on a PHP process. + By having the test groups output in XML and run in + separate processes, the output can be reparsed to + aggregate the results into a much smaller footprint top level + test. +

+ Because the XML output can come from anywhere, this opens + up the possibility of aggregating test runs from remote + servers. + A test case already exists to do this within the SimpleTest + framework, but it is currently experimental... +

+<?php
+require_once('../remote.php');
+require_once('../reporter.php');
+    
+$test_url = ...;
+$dry_url = ...;
+    
+$test = &new TestSuite('Remote tests');
+$test->addTestCase(new RemoteTestCase($test_url, $dry_url));
+$test->run(new HtmlReporter());
+?>
+

+ The RemoteTestCase takes the actual location + of the test runner, basically a web page in XML format. + It also takes the URL of a reporter set to do a dry run. + This is so that progress can be reported upward correctly. + The RemoteTestCase can be added to test suites + just like any other group test. +

+ +

Test reporter documentation

Reporting results in HTML

File test

File test

Extending the reporter

The command line reporter

Remote testing