How to run and create tests....

In creating tests there are a few goals to strive for.

1) Increase coverage of the code to help detect defects

2) Don't mess up the tester's directories. If a test wants to write out a
   file, first make sure the directory is writable, and if it is, then write
   out the files, and then delete them. So that the directory is left in its
   original state.

3) Don't require excessive disk space or CPU time. Most tests that run on a
   512 cubed volume will work just as well on a 64 cubed volume. The 64 cubed
   volume will require less disk space for the data, less CPU time to
   process, and much less CPU time to purify. The same applies to valid
   images. Typically there isn't any need for a 600 by 600 pixel valid
   image. A 300 by 300 image will work fine and require less storage and less
   CPU time to compute the image difference.

4) Try to use existing data files. If you can create a test using a data file
   that is already present, use that data file. Don't add a new data file
   unless it is critical to test the functionality you want tested. If you do
   need to add a new data file consider its size. If it is a large data file
   can you produce a lower resolution version of it and check that in instead?

The current status is that the new tree has about 100 tests and is achieving
about 43 percent coverage. These tests require about three minutes to run on
a 1 GHz Pentium and currently require about ten megabytes of disk space for
the data files and valid images. 

How to create a new test
---------------------------

The easiest approach is to look at an existing test and use that as a
starting point. The tests are kept in Testing sub-directories under each main
package. (e.g. VTK/Graphics/Testing) In Testing there will be sub-directories
for each type of test (Cxx Tcl Python Java). For C++ tests you need to write
the test and then add it to the CMakeLists.txt file in Testing/Cxx. For
Tclexamples you would write a Tcl script, and then add it to the
CMakeLists.txt file in Testing/Tcl. Just follow the examples already in the
tree. Most tests take a few common arguments:

  -D /path/to/VTKData
  -V relative/path/to/valid/image
  
Tcl scripts also accept:

  -A /path/to/vtktcl/package

Typically these arguments are optional and the test will try to run without
them. If the -V option is provided the test will try to perform image
differences between the valid image and the rendered image. If the valid
image specified with -V doesn't exist it will create it.

Running a Tcl example
-----------------------

The most common way to run a Tcl example is to first set TCLLIBPATH to
/path/to/VTK/Wrapping/Tcl so that the examples can find the vtktcl
package. On UNIX you can do:

setenv TCLLIBPATH /home/you/VTK/Wrapping/Tcl

or on windows set TCLLIBPATH to c:/your/VTK/Wrapping/Tcl note that UNIX style slashes are used on windows as well. Once TCLLIBPATH is set you can run the example as

cd VTK/Graphics/Testing/Tcl
/path/to/VTK/bin/vtk yourScript.tcl 

It will look for VTKData to be at the same level as VTK on your disk. If it
isn't you will need to provide the -D argument.

/path/to/VTK/bin/vtk yourScript.tcl -D /path/to/VTKData

to run the example as a regression test you should make the example the first
argument to VTK/Common/Testing/Tcl/rtImageTest.tcl. For example

/path/to/VTK/bin/vtk /path/to/VTK/Common/Testing/Tcl/rtImageTest.tcl yourScript.tcl -D /path/to/VTKData -V Baseline/Graphics/yourScript.png

That is a long command line but normally Dart will take care of it for you.


Producing a dashboard
-------------------------

If you want to test your build of VTK you can produce a dashboard. To do this
you will need to have Dart installed on your system. Dart can be obtained
currently using cvs as :pserver:anonymous@public.kitware.com:/cvsroot/Dart
login with a password of dart. The when you run CMake, make sure DART_ROOT is
set in your cache and BUILD_TESTING is on. Then you can do

make Experimental

on UNIX to perform a build, run the tests, and submit the result to the dashboard. If you just want to run the tests you can do 

make ExperimentalTests

on windows there are targets in the workspace for Experiemental and
ExperimentalTests






