Skip to content
Snippets Groups Projects

Test Suite

Last edited by Aaron Bray

Here we will discuss the basics of running the test suite. A script is provided to run all of our tests, utilities, and tools. We provide a simple run batch/shell script wrapper around a CMake file to ensure cross platform compatibility. CMake should be available on your path to use this tool. To call this script, your terminal must be in the <pulse/build>/install/bin, which we will refer to as <pulse/bin> and calling format to be run is as follows :

# On Windows
> run [command]
# On Linux/Mac
# Make sure to chmod to run this script on your machine
$ chmod +x run.sh
# Once done, you may execute any command
$ ./run.sh [command]

The followings [command]'s are used to generate data:

  • genData- Generate data files needed for stabilizing a patient. This reads the <pulse/source>/data/Data.xlxs file (containing Patients, Substances, etc.) and generates files into several directories in the <pulse/bin> directory.

  • genStates- Generate a state file for every patient in the <pulse/bin/patients> directory. A new state file is saved at the point where the patient completes stabilization (Simulation time 0s) to the <pulse/bin>/states directory

  • updateBaselines- Pull log and CSV files for all verifiation tests. These CSV files are the expected results of all tests. These files can change frequently. When you are developing on Pulse and pull from the repository, you should also run this command to ensure you are comparing your generated results with the latest baselines. Specifically if anything changes under this folder

  • protoc- This will recompile all google proto files. When you are developing on Pulse and modify a proto file, you can run this command to ensure your changes work. If you pull new proto file updates from the repository, the CMake ZERO_CHECK step will detect new proto files and automatically run the protoc compiler.

  • jar- This will rebuild the Pulse.jar file

  • setup.py- This copies all needed python files to <pulse/install/python> so you can just add that directory to your PYTHONPATH if desired.

  • doxygen- Generate the html we use for our website. You will need to have your system properly configured for this to work

The followings [command]'s are used to execute several utility tools:

  • rebase- Used to generate new baselines for tests. More infor can be found here

  • fullreport- This generates one html report file from all html reports files.

  • plotter- Takes in a CSV file and generates a png line plot for each header in the CSV file where the header is the Y-axis vs Time is the X-axis

  • compare- Takes in two CSV files and generates a png line plot for each header in the first CSV file. The header is the Y-axis vs Time is the X-axis. Each png line plot will have 2 lines, 1 from each CSV (if the CSV 2 does not have the header from CSV 1, the plot will only have 1 line)

  • plot- This will process a plot json file and create all plots it specifies.

The followings [command]'s are used to execute our test suite:

  • SystemValidation- Will use <pulse/source>/test/config/ValidationSystems.config which will create a CSV file per system. Next, each CSV file will be read in by the test suite, and it will compute and compare the min/max/mean of values and check how close they are to valid values provided in our sytem validation spreadsheet (<pulse/source/data/human/adult/validation/SystemValidationData.xlsx) and create system validation tables in the <pulse/bin>/test_results directory

  • PatientValidation- Will use <pulse/source>/test/config/ValidationPatients.config which will create a CSV file per patient. Next, each CSV file will be read in by the test suite, and it will compute and compare the min/max/mean of values and check how close they are to valid values provided in our patient validation spreadsheet (<pulse/source/data/human/adult/validation/PatientValidationData.xlsx) and create patioent validation tables in the <pulse/bin>/test_results directory

  • The names of any *.config file in the <pulse/source>/test/config/ directory (don't include the extension)

    • For Example : DebugRun, CDMUnitTests, ValidationSystems, VerificationScenarios, etc.
    • With exception of the Characterize.config, this is a legacy testing configuration that will not work

  • Tests- This will run all tests available in Pulse, this may take several hours depending on your system. This will also use all cores on your system.

More info on running our test suite can be found here

Comments

    Please register or sign in to add a comment.