CONTRIBUTING.md 22 KB
Newer Older
1
# Contributing to VTK-m #
2

3
This page documents how to develop VTK-m through [Git](http://git-scm.com).
4

5
Git is an extremely powerful version control tool that supports many
6
7
8
9
different "workflows" for individual development and collaboration. Here we
document procedures used by the VTK-m development community. In the
interest of simplicity and brevity we do *not* provide an explanation of
why we use this approach.
10
11


12
## Setup ##
13
14
15

Before you begin, perform initial setup:

16
1.  Register [GitLab Access] to create an account and select a user name.
17

18
2.  [Fork VTK-m] into your user's namespace on GitLab.
19

20
3.  Use Git to create a local clone of the main VTK repository:
21

22
        $ git clone https://gitlab.kitware.com/vtk/vtk-m.git
23
        $ cd vtk-m
24

25
26
    The main repository will be configured as your `origin` remote.

27
4.  Run the developer setup script to prepare your VTK-m work tree and
28
29
30
31
32
33
34
    create Git command aliases used below:

        $ ./Utilities/SetupForDevelopment.sh

    This will prompt for your GitLab user name and configure a remote
    called `gitlab` to refer to it.

35
5. (Optional but highly recommended.)
36
37
38
39
40
    [Register with the VTK-m dashboard] on Kitware's CDash instance to
    better know how your code performs in regression tests. After
    registering and signing in, click on "All Dashboards" link in the upper
    left corner, scroll down and click "Subscribe to this project" on the
    right of VTK-m.
41

42
6.  (Optional but highly recommended.)
43
44
    [Sign up for the VTK-m mailing list] to communicate with other
    developers and users.
45

46
[GitLab Access]: https://gitlab.kitware.com/users/sign_in
47
[Fork VTK-m]: https://gitlab.kitware.com/vtk/vtk-m/-/forks/new
48
[Register with the VTK-m dashboard]: https://open.cdash.org/register
49
50
[Sign up for the VTK-m mailing list]: http://vtk.org/mailman/listinfo/vtkm

51

52
## Workflow ##
53

54
55
VTK-m development uses a [branchy workflow] based on topic branches. Our
collaboration workflow consists of three main steps:
56
57

1.  Local Development:
58
59
      * [Update](#update)
      * [Create a Topic](#create-a-topic)
60

61
62
63
64
65
66
2.  Code Review (requires [GitLab Access]):
      * [Share a Topic](#share-a-topic)
      * [Create a Merge Request](#create-a-merge-request)
      * [Review a Merge Request](#review-a-merge-request)
      * [Reformat a Topic](#reformat-a-topic)
      * [Revise a Topic](#revise-a-topic)
67
68

3.  Integrate Changes:
69
      * [Merge a Topic](#merge-a-topic) (requires permission in GitLab)
70
71
72

[branchy workflow]: http://public.kitware.com/Wiki/Git/Workflow/Topic

73
74

## Update ##
75
76

1.  Update your local `master` branch:
77

78
79
        $ git checkout master
        $ git pull
80

81
2.  Optionally push `master` to your fork in GitLab:
82

83
84
85
        $ git push gitlab master


86
87
88
89
90
## Create a Topic ##

All new work must be committed on topic branches. Name topics like you
might name functions: concise but precise. A reader should have a general
idea of the feature or fix to be developed given just the branch name.
91
92
93
94
95
96
97
98
99
100

1.  To start a new topic branch:

        $ git fetch origin

2.  For new development, start the topic from `origin/master`:

        $ git checkout -b my-topic origin/master

3.  Edit files and create commits (repeat as needed):
101
102
103
104
105

        $ edit file1 file2 file3
        $ git add file1 file2 file3
        $ git commit

106
    Caveats:
107
108
109
      * Data files must be placed under a folder explicitly named 'data'.
        This is required as VTK-m uses Git-LFS to efficiently support data
        files.
110

111
112
113
4.  If you are adding a new feature or making sigificant changes to API,
    make sure to add a entry to `docs/changelog`. This allows release
    notes to properly capture all relevant changes.
114

115
### Guidelines for Commit Messages ###
116

117
118
119
Remember to *motivate & summarize*. When writing commit messages. Get into
the habit of creating messages that have enough information for any
developer to read and glean relevant information such as:
120
121
122
123
124
125

1.  Is this change important and why?
2.  If addressing an issue, which issue(s)?
3.  If a new feature, why is it useful and/or necessary?
4.  Are there background references or documentation?

126
127
128
129
130
131
132
A short description of what the issue being addressed and how will go a
long way towards making the log more readable and the software more
maintainable. VTK-m requires that your message start with a single subject
line, followed by a blank line, followed by the message body which contains
the more detailed explanatory text for the commit. You can consider a
commit message to very similar to an email with the first line being the
subject of an email and the rest of the text as the body.
133

134
Style guidelines for commit messages are as follows:
135

136
137
138
139
140
141
142
143
144
1.   Separate subject from body with a blank line
2.   Limit the subject line to 78 characters
3.   Capitalize the subject line
4.   Use the imperative mood in the subject line e.g. "Refactor foo" or
     "Fix Issue #12322", instead of "Refactoring foo", or "Fixing issue
     #12322".
5.   Wrap the body at 80 characters
6.   Use the body to explain `what` and `why` and if applicable a brief
    `how`.
145
146


147
148
149
150
151
152
153
## Share a Topic ##

When a topic is ready for review and possible inclusion, share it by
pushing to a fork of your repository in GitLab. Be sure you have registered
and signed in for [GitLab Access] and created your fork by visiting the
main [VTK-m GitLab] repository page and using the "Fork" button in the
upper right.
154
155
156
157
158
159
160
161

[VTK-m GitLab]: https://gitlab.kitware.com/vtk/vtk-m

1.  Checkout the topic if it is not your current branch:

        $ git checkout my-topic

2.  Push commits in your topic branch to your fork in GitLab:
162

163
        $ git gitlab-push
164

165
    Notes:
166
167
168
169
170
      * If you are revising a previously pushed topic and have rewritten
        the topic history, add `-f` or `--force` to overwrite the
        destination.
      * The `gitlab-push` script also pushes the `master` branch to your
        fork in GitLab to keep it in sync with the upstream `master`.
171
172
173
174
      * If you have created or modified Git-LFS files, you will need to
        push them separately using `git lfs push --all origin`. You will
        need developer access to the VTK-m repository do this. If you do
        not have developer access, ask the VTK-m group for help.
175
176
177

    The output will include a link to the topic branch in your fork in
    GitLab and a link to a page for creating a Merge Request.
178
179


180
## Create a Merge Request ##
181

182
183
When you [pushed your topic branch](#share-a-topic), it will provide you
with a url of the form
184

185
    https://gitlab.kitware.com/<username>/vtk-m/-/merge_requests/new
186

187
188
189
190
191
192
193
You can copy/paste that into your web browser to create a new merge
request. Alternately, you can visit your fork in GitLab, browse to the
"**Merge Requests**" link on the left, and use the "**New Merge Request**"
button in the upper right.

Once at the create merge request page, follow these steps. Many of these
will be filled out for you.
194
195
196
197
198

1.  In the "**Source branch**" box select the `<username>/vtk-m` repository
    and the `my-topic` branch.

2.  In the "**Target branch**" box select the `vtk/vtk-m` repository and
199
    the `master` branch. It should be the default.
200

201
202
3.  Use the "**Compare branches**" button to proceed to the next page and
    fill out the merge request creation form.
203
204

4.  In the "**Title**" field provide a one-line summary of the entire
205
    topic. This will become the title of the Merge Request.
206
207
208
209
210

    Example Merge Request Title:

        Add OpenMP Device Adapter

211
212
5.  In the "**Description**" field provide a high-level description of the
    change the topic makes and any relevant information about how to try
213
    it.
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
    *   Use `@username` syntax to draw attention of specific developers.
        This syntax may be used anywhere outside literal text and code
        blocks.  Or, wait until the [next step](#review-a-merge-request)
        and add comments to draw attention of developers.
    *   Optionally use a fenced code block with type `message` to specify
        text to be included in the generated merge commit message when the
        topic is [merged](#merge-a-topic).

    Example Merge Request Description:

        This branch adds a new device adapter that uses new OpenMP 4+ features
        including Task groups to better handle unbalanced and irregular domains

        ```message
        Add a OpenMP 4+ task-based device adapter.
        ```

        Cc: @user1 @user2

233
234
6.  The "**Assign to**", "**Milestone**", and "**Labels**" fields may be
    left blank.
235

236
237
238
239
7.  Enable the "**Allow commits from members who can merge to the target branch.**" option,
    so that reviewers can modify the merge request. This allows reviewers to change
    minor style issues without overwhelming the author with change requests.

240
241
242
7.  Use the "**Submit merge request**" button to create the merge request
    and visit its page.

243
### Guidelines for Merge Requests ###
244

245
246
247
Remember to *motivate & summarize*. When creating a merge request, consider
the reviewers and future perusers of the software. Provide enough
information to motivate the merge request such as:
248
249
250
251
252
253

1.  Is this merge request important and why?
2.  If addressing an issue, which issue(s)?
3.  If a new feature, why is it useful and/or necessary?
4.  Are there background references or documentation?

254
255
256
257
Also provide a summary statement expressing what you did and if there is a
choice in implementation or design pattern, the rationale for choosing a
certain path. Notable software or data features should be mentioned as
well.
258

259
260
261
262
263
264
A well written merge request will motivate your reviewers, and bring them
up to speed faster. Future software developers will be able to understand
the reasons why something was done, and possibly avoid chasing down dead
ends, Although it may take you a little more time to write a good merge
request, you’ll likely see payback in faster reviews and better understood
and maintainable software.
265
266


267
## Review a Merge Request ##
268
269

Add comments mentioning specific developers using `@username` syntax to
270
271
272
draw their attention and have the topic reviewed. After typing `@` and some
text, GitLab will offer completions for developers whose real names or user
names match.
273

274
275
276
Comments use [GitLab Flavored Markdown] for formatting. See GitLab
documentation on [Special GitLab References] to add links to things like
merge requests and commits in other repositories.
277
278
279
280
281
282
283
284
285
286
287

[GitLab Flavored Markdown]: https://gitlab.kitware.com/help/markdown/markdown
[Special GitLab References]: https://gitlab.kitware.com/help/markdown/markdown#special-gitlab-references

### Reviews ###

Reviewers may add comments providing feedback or to acknowledge their
approval. All comments use the [GitLab Flavored Markdown][], any line of a
comment may be exactly one of the following votes followed by nothing but
whitespace before the end of the line:

288
289
  * `-1` or :-1: (`:-1:`) means "The change is not ready for integration."
  * `+1` or :+1: (`:+1:`) means "The change is ready for integration."
290
291

These are used to inform the author that a merge srequest has been approved
292
for [merging](#merge-a-topic).
293

294
#### Fetching Changes ####
295

296
297
298
One may fetch the changes associated with a merge request by using the `git
fetch` command line shown at the top of the Merge Request page. It is of
the form:
299
300
301
302
303
304
305

    $ git fetch https://gitlab.kitware.com/$username/vtk-m.git $branch

This updates the local `FETCH_HEAD` to refer to the branch.

There are a few options for checking out the changes in a work tree:

306
  * One may checkout the branch:
307
308

        $ git checkout FETCH_HEAD -b $branch
309

310
311
312
313
    or checkout the commit without creating a local branch:

        $ git checkout FETCH_HEAD

314
  * Or, one may cherry-pick the commits to minimize rebuild time:
315
316
317
318
319

        $ git cherry-pick ..FETCH_HEAD

### Robot Reviews ###

320
321
322
323
The "Kitware Robot" automatically performs basic checks on the commits and
adds a comment acknowledging or rejecting the topic. This will be repeated
automatically whenever the topic is updated. A re-check may be explicitly
requested by adding a comment with a single *trailing* line:
324
325
326
327
328
329
330
331

    Do: check

A topic cannot be [merged](#merge-a-topic) until the automatic review
succeeds.

### Testing ###

332
333
334
335
336
337
338
339
340
341
342
343
344
345
Each time a merge request is created or updated automated testing
is automatically triggered, and shows up under the pipeline tab.

Developers can track the status of the pipeline for a merge
request by using the Pipeline tab on a merge request or by
clicking on stage icons as shown below:

![alt text](docs/build_stage.png "Pipeline")

When trying to diagnose why a build or tests stage has failed it
generally is easier to look at the pruned information reported
on [VTK-m's CDash Dashboard](https://open.cdash.org/index.php?project=VTKM).
To make it easier to see only the results for a given merge request
you can click the `cdash` link under the external stage ( rightmost pipeline stage icon )
346

347
348
349
350
351
352
353
354
355
356
![alt text](docs/external_stage.png "CDash Link")

In addition to the gitlab pipelines the buildbot user (@buildbot) will respond
with a comment linking to the CDash results when it schedules builds.

The builds for VTK-m that show up as part of the `external` stage of the
gitlab pipeline are driven via buildbot, and have a different workflow.
When you need to do things such as retry a build, you must issue commands
via comments of the following form. The buildbot user (@buildbot) will
respond signify that the command has been executed
357
358
359
360
361

    Do: test

The `Do: test` command accepts the following arguments:

362
  * `--oneshot`
363
364
        only build the *current* hash of the branch; updates will not be
        built using this command
365
  * `--stop`
366
        clear the list of commands for the merge request
367
  * `--superbuild`
368
        build the superbuilds related to the project
369
  * `--clear`
370
        clear previous commands before adding this command
371
  * `--regex-include <arg>` or `-i <arg>`
372
373
        only build on builders matching `<arg>` (a Python regular
        expression)
374
  * `--regex-exclude <arg>` or `-e <arg>`
375
376
377
        excludes builds on builders matching `<arg>` (a Python regular
        expression)

378
379
380
Multiple `Do: test` commands may be given in separate comments. Buildbot
may skip tests for older branch updates that have not started before a test
for a new update is requested.
381

382
Build names always follow this pattern:
383

384
        SHA-build#-[os-libtype-buildtype+feature1+feature2]-topic
385

386
387
  * SHA: The shortened 8-digit SHA identifying the git commit being tested
  * build: `build####` with `####` replaced by a unique number for the build
388
389
390
391
  * os: one of `windows`, `osx`, or `linux`
  * libtype: `shared` or `static`
  * buildtype: `release` or `debug`
  * feature: alphabetical list of features enabled for the build
392
  * topic: the git topic branch being tested
393
394


395
## Reformat a Topic ##
396

397
398
399
400
401
The "Kitware Robot" automatically performs basic code formatting on the
commits and adds a comment acknowledging or rejecting a merge request based
on the format. You may request "Kitware Robot" to automatically reformat
the remote copy of your branch by issuing the following command in a merge
request page comment:
402
403
404
405
406

    Do: reformat

This reformatting of the topic rewrites the commits to fix the formatting
errors, and causes the version on the developers machine to differ from
407
408
409
version on the gitlab server. To resolve this issue you must update the
local version to match the reformatted one on the server if you wish to
extend or revise the topic.
410
411
412
413
414
415

1.  Checkout the topic if it is not your current branch:
        $ git checkout my-topic

2.  Get the new version from gitlab

Kenneth Moreland's avatar
Kenneth Moreland committed
416
        $ git gitlab-sync
417
418
419


If you do not wish to have the "Kitware Robot" automatically reformat your
420
421
422
branch you can do so manually by running [clang-format] manually on each
commit of your branch. This must be done by [revising each
commit](#revise-a-topic) not as new commits onto the end of the branch.
423

424
[clang-format]: https://clang.llvm.org/docs/ClangFormat.html
425

426

427
428
429
430
431
432
433
434
## Revise a Topic ##

Revising a topic is a special way to modify the commits within a topic.
Normally during a review of a merge request a developer will resolve issues
brought up during review by adding more commits to the topic. While this is
sufficient for most issues, some issues can only be resolved by rewriting
the history of the topic.

435
### Starting Revisions ###
436

437
438
Regardless of what revisions need to be made, you first must make sure that
your topic is the current branch. To make your topic branch current:
439

440
    $ git checkout my-topic
441

442
(As always, you can get the current branch with the `git status` command.)
443

444
445
446
447
448
A common need for revisions is to update your topic branch to the latest
version of VTK-m. Even if you a revising your topic branch for some other
reason, also updating to the latest master is usually not a bad idea. To
update to the latest commit in master, you need to make sure that the
latest commit is in your local repository. To do that run
449

450
    $ git pull-master
451

452
### Make Local Revisions ###
453

454
455
The easiest way to make changes to the commits on your topic branch is to
rebase it to the master branch:
456

457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
    $ git rebase -i master

When you run this command, git will open your text editor with a list of
all the commits that will be changed. The first word of each lines
indicates a command for that commit. By default, all commits are `pick`ed,
which means that they will be simply passed without change.

If you need to revise the commit message of one of the commits, then
replace `pick` with `reword` (or just `r`). When the rebase starts, a new
editor will be provided to let you change the commit message.

If you need to make changes to files within a commit, then replace `pick`
with `edit` (or just `e`). When the rebase gets to this commit, it will
pause to let you make changes to the files.

If you need to merge commits together, use the `squash` (or `s`) command.
The rebase will give you a change to edit the commit message of the merged
commit.

Once you exit your editor, the rebase will begin. If you have requested to
edit any commits or if git detects a conflict while applying a commit, it
will stop running so that you can make changes. Make the changes you need,
use `git add` to stage those changes, and then use

    $ git rebase --continue
482

483
484
485
486
487
488
489
490
491
492
493
494
to have git continue the rebase process. You can always run `git status` to
get help about what to do next.

### Push to GitLab ###

To push commits in your topic branch to your fork in GitLab:

    $ git gitlab-push -f

Note: You need have the `-f` or `--force` to overwrite the destination as
you are revising a previously pushed topic and have rewritten the topic
history.
495

496
## Merge a Topic ##
497
498

After a topic has been reviewed and approved in a GitLab Merge Request,
499
authorized developers may add a comment with a single *trailing* line:
500
501
502

    Do: merge

503
to ask that the change be merged into the upstream repository. By
luz.paz's avatar
luz.paz committed
504
convention, only merge if you have received `+1` . Do not request a merge
505
if any `-1` review comments have not been resolved.
506
507
508
509
510
511
512
513
514

### Merge Success ###

If the merge succeeds the topic will appear in the upstream repository
`master` branch and the Merge Request will be closed automatically.

### Merge Failure ###

If the merge fails (likely due to a conflict), a comment will be added
515
describing the failure. In the case of a conflict, fetch the latest
516
517
518
519
520
521
upstream history and rebase on it:

    $ git fetch origin
    $ git rebase origin/master

Return to the [above step](#share-a-topic) to share the revised topic.
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586

## Fixing Problems ##

There are a lot of instructions in this document, and if you are not
familiar with contributing to VTK-m, you may get your repository in a bad
state that will cause problems with the other instructions. This section
attempts to capture common problems contributors have and the fixes for
them.

### Wrong origin Remote ###

The VTK-m contribution workflow assumes that your `origin` remote is
attached to the main VTK-m GitLab repository. If it is not, that will cause
problems with updating your repository. To check which remote repository
origin refers to, run

    $ git remote -v

It will give you a list of remotes and their URLs that you have configured.
If you have a line like

    origin  https://gitlab.kitware.com/vtk/vtk-m.git (fetch)

or

    origin  git@gitlab.kitware.com:vtk/vtk-m.git (fetch)

then everything is OK. If it is anything else (for example, it has your
GitLab username in it), then you have a problem. Fortunately, you can fix
it by simply changing the remote's URL:

    $ git remote set-url origin https://gitlab.kitware.com/vtk/vtk-m.git

If you had to change the `origin` remote, you should also rerun
`Utilities/SetupForDevelopment.sh` to make sure the other remotes are set
up correctly.

### master Not Tracking origin ###

The instructions in this document assume that your `master` branch is
tracking the remote `master` branch at `origin` (which, as specified above,
should be the main VTK-m repository). This should be set up if you
correctly cloned the main VTK-m repository, but can get accidentally
changed.

To check which remote branch `master` is tracking, call

    $ git rev-parse --abbrev-ref --symbolic-full-name master@{upstream}

Git should respond with `origin/master`. If it responds with anything else,
you need to reset the tracking:

    $ git branch -u origin/master master

### Local Edits on the master Branch ###

The first step in the [contributing workflow](#workflow) is that you
[create a topic branch](#create-a-topic) on which to make changes. You are
not supposed to add your commits directly to `master`. However, it is easy
to forget to create the topic branch.

To find out if you have local commits on your master branch, check its
status:

    $ git checkout master
587
    $ git status
588
589
590
591
592
593
594
595
596
597
598
599
600
601

If status responds that your branch is up to date or that your branch is
_behind_ the `origin/master` remote branch, then everything is fine. (If
your branch is behind you might want to update it with `git pull`.)

If the status responds that your branch and `origin/master` have diverged
or that your branch is _ahead_ of `origin/master`, then you have local
commits on the master branch. Those local commits need to move to a topic
branch.

1.  Create a topic branch:

        $ git branch my-topic

602
    Of course, replace `my-topic` with something that better describes your
603
604
605
606
607
608
609
610
611
    changes.

2.  Reset the local master branch to the remote master branch:

        $ git reset --hard origin/master

3.  Check out the topic branch to continue working on it:

        $ git checkout my-topic