Enhance validation process
The command validate-system doesn't currently include the validation of the self-reported outputs
Steps to reproduce
Well just run validate-system and see by yourself
What is the expected correct behavior?
The validate-system command should validate three things:
- That the
container_outputsdirectory contains at least one system output,
- That each reported system output is different from the baseline's,
- The directory name starts with a dataset name
These three checks should be performed all the time, and only cause a failure if a
--strict flag is provided.
Look at this, it contains the function to compare two system outputs: https://gitlab.kitware.com/alexandreB/indus-config/blob/master/step_actev_validate_nist_execution.sh
Move the baseline system output from container_outputs to bin/...