Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
VTK
VTK
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 581
    • Issues 581
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 162
    • Merge Requests 162
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • VTK
  • VTKVTK
  • Issues
  • #18113

Closed
Open
Opened Feb 04, 2021 by Ben Boeckel@ben.boeckelOwner0 of 3 tasks completed0/3 tasks

CI jobs for contract testing

VTK has a complex API that breaks occasionally without warning. In addition, behaviors sometimes change without warning. In order to make VTK developers aware of when these changes matter to "significant" projects, VTK should run contract testing for these using a successful build of VTK itself.

Projects of note (can be expanded):

  • VTK's own test suite (see Testing/External) (!7653 (merged))
  • ParaView (needs support for an external VTK to be in the repository, see paraview/paraview#14157 (closed))
  • Slicer (IIRC, needs support for a pre-built VTK)

The process would be to take a VTK CI job that has passed testing, install it to a directory and upload it as an artifact. Then each project can take this artifact, extract it into an environment where the project-under-test is cloned, built, and tested against the installed VTK.

Projects should always build, but testing is harder to quantify in addition to taking a long time. CDash results can be uploaded to both VTK's CDash instance and the main project's instance (see this file from SMTK for how to submit to multiple CDash instances). In order to subset the test suite of contract projects, tests can gain a CTest label which VTK can use to run that subset. If wanted, the full set can then be run and uploaded to the project's main CDash instance (excluding the tests already run for VTK).

Cc: @jcfr @utkarsh.ayachit

Edited Feb 19, 2021 by Ben Boeckel
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: vtk/vtk#18113