develop.md 19.4 KB
Newer Older
1
2
Develop ParaView with Git
=========================
3

4
This page documents how to develop ParaView through [Git][].
5
6
7
8
9
10
See the [README](README.md) for more information.

[Git]: http://git-scm.com

Git is an extremely powerful version control tool that supports many
different "workflows" for individual development and collaboration.
11
Here we document procedures used by the ParaView development community.
12
13
14
15
16
17
18
19
In the interest of simplicity and brevity we do *not* provide an
explanation of why we use this approach.

Setup
-----

Before you begin, perform initial setup:

20
1.  Register [GitLab Access][] to create an account and select a user name.
21

22
2.  [Fork ParaView][] into your user's namespace on GitLab.
23

24
3.  Follow the [download instructions](download.md#clone) to create a
25
26
27
    local clone of the main ParaView repository.  Optionally configure
    Git to [use SSH instead of HTTPS](download.md#use-ssh-instead-of-https).
    Then clone:
28

29
        $ git clone --recursive https://gitlab.kitware.com/paraview/paraview.git ParaView
30
        $ cd ParaView
31
    The main repository will be configured as your `origin` remote.
32

33
4.  Run the [developer setup script][] to prepare your ParaView work tree and
34
35
36
    create Git command aliases used below:

        $ ./Utilities/SetupForDevelopment.sh
37
38
    This will prompt for your GitLab user name and configure a remote
    called `gitlab` to refer to it.
39

40
5.  (Optional but highly recommended.)
41
    [Register](https://open.cdash.org/register.php) with the ParaView project
42
43
44
    on Kitware's CDash instance to better know how your code performs in
    regression tests.  After registering and signing in, click on
    "All Dashboards" link in the upper left corner, scroll down and click
45
    "Subscribe to this project" on the right of ParaView.
46
47

[GitLab Access]: https://gitlab.kitware.com/users/sign_in
48
[Fork ParaView]: https://gitlab.kitware.com/paraview/paraview/-/forks/new
49
50
51
52
53
[developer setup script]: /Utilities/SetupForDevelopment.sh

Workflow
--------

54
ParaView development uses a [branchy workflow][] based on topic branches.
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
Our collaboration workflow consists of three main steps:

1.  Local Development:
    * [Update](#update)
    * [Create a Topic](#create-a-topic)

2.  Code Review (requires [GitLab Access][]):
    * [Share a Topic](#share-a-topic)
    * [Create a Merge Request](#create-a-merge-request)
    * [Review a Merge Request](#review-a-merge-request)
    * [Revise a Topic](#revise-a-topic)

3.  Integrate Changes:
    * [Merge a Topic](#merge-a-topic) (requires permission in GitLab)
    * [Delete a Topic](#delete-a-topic)

[branchy workflow]: http://public.kitware.com/Wiki/Git/Workflow/Topic

Update
------

1.  Update your local `master` branch:

        $ git checkout master
79
        $ git pullall
80
81
82
83
84
2.  Optionally push `master` to your fork in GitLab:

        $ git push gitlab master
    to keep it in sync.  The `git gitlab-push` script used to
    [Share a Topic](#share-a-topic) below will also do this.
85
86
87
88
89
90

Create a Topic
--------------

All new work must be committed on topic branches.
Name topics like you might name functions: concise but precise.
Utkarsh Ayachit's avatar
Utkarsh Ayachit committed
91
92
93
94
A reader should have a general idea of the feature or fix to be developed given
just the branch name. Additionally, it is preferred to have an issue associated with
every topic. The issue can document the bug or feature to be developed. In such
cases, being your topic name with the issue number.
95
96
97
98
99

1.  To start a new topic branch:

        $ git fetch origin

Utkarsh Ayachit's avatar
Utkarsh Ayachit committed
100
101
102
103
    If there is an issue associated with the topic, assign the issue to yourself
    using the "**Assignee**" field, and add the
    `workflow:active-development` label to it.

104
105
106
107
108
109
110
111
112
2.  For new development, start the topic from `origin/master`:

        $ git checkout -b my-topic origin/master

    For release branch fixes, start the topic from `origin/release`, and
    by convention use a topic name starting in `release-`:

        $ git checkout -b release-my-topic origin/release

113
114
115
116
    If subdmodules may have changed, the  run:

        $ git submodule update

117
118
119
120
121
122
123
3.  Edit files and create commits (repeat as needed):

        $ edit file1 file2 file3
        $ git add file1 file2 file3
        $ git commit

    Caveats:
124
    * To add data follow these [vtk instructions][].
125
    * To add icons, Kitware's graphic designer may be able to help create an SVG icon.
126

127
128
129
130
131
132
133
134
135
    Commit messages must contain a brief description as the first line
    and a more detailed description of what the commit contains. If
    the commit contains a new feature, the detailed message must
    describe the new feature and why it is needed. If the commit
    contains a bug fix, the detailed message must describe the bug
    behavior, its underlying cause, and the approach to fix it. If the
    bug is described in the bug tracker, the commit message must
    contain a reference to the bug number.

136
137
138
139
140
141
4. Add some tests

    * Start `paraview.exe -dr` to ignore prefs (disable registry)
    * Choose `Tools .. Record Test` to start.
    * Choose `Tools .. Lock View Size Custom...` - a 400x400 window works well.
    * Perform actions in the GUI that exercise your feature. Stop recording.
142
    * Put the resulting XML file into `Clients/ParaView/Testing/XML`
143
144
145
    * Add it to CMakeLists.txt, probably in a TESTS_WITH_BASELINES section
        * you can manually add `<pqcompareview>` for multiple image comparisons, then add to the TESTS_WITH_INLINE_COMPARES section
    * Follow the [vtk instructions][] to add the baseline images, which live in `Testing/Data/Baseline/`.
146
        * Add your new baseline images to the list in `Testing/XML/CMakeLists.txt`
147
148
149
150
    * Add all testing files to your topic.

    Some background is in the [testing design wiki](https://www.paraview.org/Wiki/Testing_design).

151
152
5. Add release notes

luz paz's avatar
luz paz committed
153
    Notable changes should create a new file in `Documentation/release/dev/`
154
155
156
157
    named `my-topic.md` (replace `my-topic` with the name of your topic).
    This is not necessary for branches which are "trivial" such as fixing
    typos, updating test baselines, or are developer-oriented.

158
[vtk instructions]: https://gitlab.kitware.com/vtk/vtk/-/blob/master/Documentation/dev/git/data.md
159

160
161
162
163
164
165
Share a Topic
-------------

When a topic is ready for review and possible inclusion, share it by pushing
to a fork of your repository in GitLab.  Be sure you have registered and
signed in for [GitLab Access][] and created your fork by visiting the main
166
[ParaView GitLab][] repository page and using the "Fork" button in the upper right.
167

168
[ParaView GitLab]: https://gitlab.kitware.com/paraview/paraview
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184

1.  Checkout the topic if it is not your current branch:

        $ git checkout my-topic

2.  Check what commits will be pushed to your fork in GitLab:

        $ git prepush

3.  Push commits in your topic branch to your fork in GitLab:

        $ git gitlab-push

    Notes:
    * If you are revising a previously pushed topic and have rewritten the
      topic history, add `-f` or `--force` to overwrite the destination.
185
    * If the topic adds data see [this note](https://gitlab.kitware.com/vtk/vtk/-/blob/master/Documentation/dev/git/data.md#push).
186
187
    * The `gitlab-push` script also pushes the `master` branch to your
      fork in GitLab to keep it in sync with the upstream `master`.
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202

    The output will include a link to the topic branch in your fork in GitLab
    and a link to a page for creating a Merge Request.

Create a Merge Request
----------------------

(If you already created a merge request for a given topic and have reached
this step after revising it, skip to the [next step](#review-a-merge-request).)

Visit your fork in GitLab, browse to the "**Merge Requests**" link on the
left, and use the "**New Merge Request**" button in the upper right to
reach the URL printed at the end of the [previous step](#share-a-topic).
It should be of the form:

203
    https://gitlab.kitware.com/<username>/paraview/-/merge_requests/new
204
205
206

Follow these steps:

207
1.  In the "**Source branch**" box select the `<username>/paraview` repository
208
209
    and the `my-topic` branch.

210
2.  In the "**Target branch**" box select the `paraview/paraview` repository and
211
212
213
214
215
216
    the `master` branch.  It should be the default.

    If your change is a fix for the `release` branch, you should still
    select the `master` branch as the target because the change needs
    to end up there too.

217
218
3.  Use the "**Compare branches**" button to proceed to the next page
    and fill out the merge request creation form.
219

220
221
222
223
224
225
4.  In the "**Title**" field provide a one-line summary of the entire
    topic.  This will become the title of the Merge Request.

    Example Merge Request Title:

        Wrapping: Add Java 1.x support
226
227
228

5.  In the "**Description**" field provide a high-level description
    of the change the topic makes and any relevant information about
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
    how to try it.
    *   Use `@username` syntax to draw attention of specific developers.
        This syntax may be used anywhere outside literal text and code
        blocks.  Or, wait until the [next step](#review-a-merge-request)
        and add comments to draw attention of developers.
    *   If your change is a fix for the `release` branch, indicate this
        so that a maintainer knows it should be merged to `release`.
    *   Optionally use a fenced code block with type `message` to specify
        text to be included in the generated merge commit message when the
        topic is [merged](#merge-a-topic).

    Example Merge Request Description:

        This branch requires Java 1.x which is not generally available yet.
        Get Java 1.x from ... in order to try these changes.
244
245

        ```message
246
        Add support for Java 1.x to the wrapping infrastructure.
247
248
        ```

249
250
251
252
253
254
255
        Cc: @user1 @user2

6.  The "**Assign to**", "**Milestone**", and "**Labels**" fields
    may be left blank.

7.  Use the "**Submit merge request**" button to create the merge request
    and visit its page.
256
257
258
259
260

Review a Merge Request
----------------------

Add comments mentioning specific developers using `@username` syntax to
261
262
263
draw their attention and have the topic reviewed.  After typing `@` and
some text, GitLab will offer completions for developers whose real names
or user names match.
264

265
266
267
268
Comments use [GitLab Flavored Markdown][] for formatting.  See GitLab
documentation on [Special GitLab References][] to add links to things
like merge requests and commits in other repositories.

Utkarsh Ayachit's avatar
Utkarsh Ayachit committed
269
270
271
272
273
274
275
276
277
When a merge request is ready for review, developers can use the
`triage:ready-for-review` to indicate the same to the reviewers. If reviewers
deems that it needs more work, they can add the `triage:needswork` label.
This can be repeated as many times as needed adding/removing labels as
appropriate.

If a merge request is waiting on dashboards, use the `triage:pending-dashboards`
label.

278
279
280
[GitLab Flavored Markdown]: https://gitlab.kitware.com/help/markdown/markdown
[Special GitLab References]: https://gitlab.kitware.com/help/markdown/markdown#special-gitlab-references

281
282
### Human Reviews ###

283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
Reviewers may add comments providing feedback or to acknowledge their
approval.  Lines of specific forms will be extracted during
[merging](#merge-a-topic) and included as trailing lines of the
generated merge commit message:

The *leading* line of a comment may optionally be exactly one of the
following votes followed by nothing but whitespace before the end
of the line:

*   `-1` or :-1: (`:-1:`) means "The change is not ready for integration."
*   `+1` or :+1: (`:+1:`) means "I like the change but defer to others."
*   `+2` means "The change is ready for integration."
*   `+3` means "I have tested the change and verified it works."

The middle lines of a comment may be free-form [GitLab Flavored Markdown][].

Zero or more *trailing* lines of a comment may each contain exactly one
of the following votes followed by nothing but whitespace before the end
of the line:

*   `Rejected-by: me` means "The change is not ready for integration."
*   `Acked-by: me` means "I like the change but defer to others."
*   `Reviewed-by: me` means "The change is ready for integration."
*   `Tested-by: me` means "I have tested the change and verified it works."

Each `me` reference may instead be an `@username` reference or a full
`Real Name <user@domain>` reference to credit someone else for performing
the review.  References to `me` and `@username` will automatically be
transformed into a real name and email address according to the user's
GitLab account profile.
313

314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
#### Fetching Changes ####

One may fetch the changes associated with a merge request by using
the `git fetch` command line shown at the top of the Merge Request
page.  It is of the form:

    $ git fetch https://gitlab.kitware.com/$username/paraview.git $branch

This updates the local `FETCH_HEAD` to refer to the branch.

There are a few options for checking out the changes in a work tree:

*   One may checkout the branch:

        $ git checkout FETCH_HEAD -b $branch
    or checkout the commit without creating a local branch:

        $ git checkout FETCH_HEAD

*   Or, one may cherry-pick the commits to minimize rebuild time:

        $ git cherry-pick ..FETCH_HEAD

337
338
339
340
341
### Robot Reviews ###

The "Kitware Robot" automatically performs basic checks on the commits
and adds a comment acknowledging or rejecting the topic.  This will be
repeated automatically whenever the topic is pushed to your fork again.
Utkarsh Ayachit's avatar
Utkarsh Ayachit committed
342
343
A re-check may be explicitly requested by adding a comment with a single
[*trailing* line](#trailing-lines):
344
345
346
347
348
349

    Do: check

A topic cannot be [merged](#merge-a-topic) until the automatic review
succeeds.

Ben Boeckel's avatar
Ben Boeckel committed
350
351
### Testing ###

352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
ParaView uses [GitLab CI][] to test merge requests, configured by the top-level
`.gitlab-ci.yml` file.  Results may be seen both on the merge request's
pipeline page and on the [ParaView CDash Page][].  Filtered CDash results
showing just the pipeline's jobs can be reached by selecting the `cdash-commit`
job in the `External` stage of the pipeline. Note that due to GitLab changes,
the `External` stage may be in a separate pipeline for the same commit.

Lint build jobs run automatically after every push. Heavier jobs require a
manual trigger to run:

* Merge request authors may visit their merge request's pipeline and click the
  "Play" button on one or more jobs manually.  If the merge request has the
  "Allow commits from members who can merge to the target branch" check box
  enabled, ParaView maintainers may use the "Play" button too.

* [ParaView GitLab Project Developers][] may trigger CI on a merge request by
  adding a comment with a command among the [comment trailing
  lines](#trailing-lines):

        Do: test

  `@kwrobot` will add an award emoji to the comment to indicate that it
  was processed and also trigger all manual jobs in the merge request's
  pipeline.

  The `Do: test` command accepts the following arguments:

  * `--named <regex>`, `-n <regex>`: Trigger jobs matching `<regex>` anywhere
    in their name.  Job names may be seen on the merge request's pipeline page.
  * `--stage <stage>`, `-s <stage>`: Only affect jobs in a given stage. Stage
    names may be seen on the merge request's pipeline page.  Note that the
    names are determined by what is in the `.gitlab-ci.yml` file and may be
    capitalized in the web page, so lowercasing the webpage's display name for
    stages may be required.
  * `--action <action>`, `-a <action>`: The action to perform on the jobs.
    Possible actions:

    * `manual` (the default): Start jobs awaiting manual interaction.
    * `unsuccessful`: Start or restart jobs which have not completed
      successfully.
    * `failed`: Restart jobs which have completed, but without success.
    * `completed`: Restart all completed jobs.

If the merge request topic branch is updated by a push, a new manual trigger
using one of the above methods is needed to start CI again. Currently running
jobs will generally be canceled automatically.

[GitLab CI]: https://gitlab.kitware.com/help/ci/README.md
[ParaView CDash Page]: https://open.cdash.org/index.php?project=ParaView
[ParaView GitLab Project Developers]: https://gitlab.kitware.com/cmake/cmake/-/settings/members
Utkarsh Ayachit's avatar
Utkarsh Ayachit committed
402

403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
Revise a Topic
--------------

If a topic is approved during GitLab review, skip to the
[next step](#merge-a-topic).  Otherwise, revise the topic
and push it back to GitLab for another review as follows:

1.  Checkout the topic if it is not your current branch:

        $ git checkout my-topic

2.  To revise the `3`rd commit back on the topic:

        $ git rebase -i HEAD~3

    (Substitute the correct number of commits back, as low as `1`.)
    Follow Git's interactive instructions.

3.  Return to the [above step](#share-a-topic) to share the revised topic.

Merge a Topic
-------------

After a topic has been reviewed and approved in a GitLab Merge Request,
authorized developers may add a comment of the form

    Do: merge

431
432
433
434
to ask that the change be merged into the upstream repository.  By
convention, do not request a merge if any `-1` or `Rejected-by:`
review comments have not been resolved and superseded by at least
`+1` or `Acked-by:` review comments from the same user.
435

436
437
438
439
Developers are encouraged to merge their own merge requests on review. However,
please do not merge unless you are available to address any dashboard issues that may
arise. Developers who repeatedly ignore dashboard issues following their merges may
loose developer privileges to the repository temporarily (or permanently)!
440

441
442
443
444
### Merge Success ###

If the merge succeeds the topic will appear in the upstream repository
`master` branch and the Merge Request will be closed automatically.
Utkarsh Ayachit's avatar
Utkarsh Ayachit committed
445
446
447
448
449
450
Any issues associated with the Merge Request will generally get closed
automatically. If not, the developer merging the changes should close such issues
and add a `workflow:customer-review` tag to the issue(s) addressed by the change.
Reporters and testers can then review the fix. Try to add enough information to
the Issue or the Merge Request to indicate how to test the functionality if not
obvious from the original Issue.
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487

### Merge Failure ###

If the merge fails (likely due to a conflict), a comment will be added
describing the failure.  In the case of a conflict, fetch the latest
upstream history and rebase on it:

    $ git fetch origin
    $ git rebase origin/master

(If you are fixing a bug in the latest release then substitute
`origin/release` for `origin/master`.)

Return to the [above step](#share-a-topic) to share the revised topic.

Delete a Topic
--------------

After a topic has been merged upstream the Merge Request will be closed.
Now you may delete your copies of the branch.

1.  In the GitLab Merge Request page a "**Remove Source Branch**"
    button will appear.  Use it to delete the `my-topic` branch
    from your fork in GitLab.

2.  In your work tree checkout and update the `master` branch:

        $ git checkout master
        $ git pull

3.  Delete the local topic branch:

        $ git branch -d my-topic

    The `branch -d` command works only when the topic branch has been
    correctly merged.  Use `-D` instead of `-d` to force the deletion
    of an unmerged topic branch (warning - you could lose commits).
488
489
490
491
492
493

Contribute VTK Changes
----------------------

If you have any VTK changes, then you are required to get your changes
incorporated into VTK using [VTK's development workflow][]. Once your VTK topic has
494
495
496
497
498
499
500
501
502
503
504
505
506
been approved and merged into VTK, then:

1. Create a [ParaView topic](#create-a-topic) if you haven't already.
2. Add your VTK topic head (or the latest VTK
   origin/master which includes your VTK topic head).

        $ cd VTK
        $ git checkout master
        $ cd ..
        $ git add VTK
        $ git commit

3. Follow the merge process documented earlier.
507

508
[VTK's development workflow]: https://gitlab.kitware.com/vtk/vtk/-/tree/master/Documentation/dev/git