These tests are provided for Xalan contributors to evaluate the impact of code changes. Run the tests on the unchanged code, make the change and rebuild. Then run the tests again and compare the results. Results are automatically compared to the files in the "-gold" directory trees. Even though not all tests have "gold" files, it's still valuable to run the tests before and after a code change. That way you can at least ensure that your changes didn't cause any regressions to the code before you check your changes in. In the future, we hope to provide a tool to compare test results from one run to another (without necessarily having to re-run the test) to make this process even simpler.
The Xalan tests include a richly featured and fully automated testing framework that the great majority of the tests use. The org.apache.qetest package provides an independent testing and automation harness for Xalan, including logging and reporting mechanisims. The basic framework is easily used in testing other programs as well since dependencies on Xalan and an XML parser are clearly compartmented.
Nearly all tests are automated, run without any user interaction and with a minimum amount of setup, and produce
a rolled-up report of their pass/fail/other status. Our existing testing library includes
a wide array of tests, from XSLT conformance to detailed API tests, and welcomes
user-submitted tests as well. The 'smoketest' target (a subset of the most important
tests) are also integrated into the
Java Test Drivers (data driven testing)
A Java Test Driver executes a test for each xml/xsl file pair in
the specified directory tree or each pair in the specified fileList.
The driver iterates over the inputDir tree or list of files
and asks a Testlet to execute() a test on each one. This is also similar to
data driven testing, where a common algorithim is defined for a test case, and
then a large number of data points (in this case, the xml/xsl file pairs) are run through the test case in order.
The best example is
The Test Drivers rely on various Testlet implementations
to define the actual testing algorithim to apply to each xml/xsl
file pair. This defines any options to be used when processing the
file as well as logging out information about the test in progress.
Examples include
The Testlets rely on
The three levels of iteration, test algorithim, and processor flavor are all independently changeable, so we can easily try out different kinds of tests. This technique is used to run the full sets of 'conf' (conformance), 'perf' (performance), extensions, and 'contrib' (user contributed stylesheets) tests.
Java API tests for the TRAX (or javax.xml.transform) interface, that
Xalan-J 2.x implements.
All in package: org.apache.qetest.trax
All in subpackages of: org.apache.qetest.trax
A few tests are ones that Xalan does not currently pass due to Bugzilla reports, but we know the correct ("gold") result by analysis or by trying the test on other processors. A number of tests may also be missing matching "gold" files, if we haven't yet had time to confirm the correct output. It's still useful to run these tests (although the test driver will report an AMBG or 'Ambiguous' result) because you can still see if the output looks basically correct, and compare the output to previous test runs before you submit your code changes, etc.
The tests have several different types of results beyond just pass or fail,
which are documented in