Tutorial

Server setup

This feature is contained in the "git-validated-merge" plugin. It is enabled by default for Jenkins Enterprise by CloudBees, but if you don’t find it, go install this plugin. Go to your CI build/test job configuration page and check "Enable Git validated merge support" Once this option is selected, you’ll see the "Git Repository for Validated Merge" item appear in the menu list of the job page:

Figure 11.1. The "Git Repository for Validated Merge" screen

repo page

Depending on how you configure your security setting, you might see a warning in this page asking you to fix the port number of the Jenkins SSHD service. If you see it, go to the system configuration page and set the SSHD port under the "SSH Server" section, as shown below.

Figure 11.2. Configuring the SSHD port

sshd port

Then, make sure that Jenkins can push to your upstream Git repository. See the section called “Pushing to the upstream repository” for more discussion.

Your server is now ready for the validated merge.

Client setup

Now, on a developer laptop where you’d like to use the validated merge feature, register this Jenkins job as the remote repository. The exact URL of the Git repository in the Jenkins job can be obtained by the "Git Repository for Validated Merge" page on the job top page. You might see an HTTP URL instead of SSH URL, if you haven’t enabled the security setting.

$ git remote add jenkins ssh://myserver:2222/foo

Instead of registering this as a separate repository, you can also set Jenkins as the push URL of your upstream repository. In this way, your push will always go through Jenkins automatically, without you typing anything differently from command line. In comparison, the previous approach would require that you consciously select Jenkins as the target of each push.

$ git remote set-url --push origin ssh://myserver:2222/foo

This needs to be repeated for every workspace, but now your client is also ready for the validated merge.

Mental picture of the validated merge

Before we go discussing how you actually do a validated merge, let’s briefly talk about the mental picture of how this whole thing works.

With the validated merge, your Jenkins job acts as an intermediate repository between your workspace and your central/upstream repository. Instead of pushing changes directory to the upstream repository, you’ll push changes to the Jenkins job. We’ll refer to this intermediate repository as the gate repository.

The gate repository implemented by Jenkins is a little bit magical. When you push a ref to the gate repository (for example via git push jenkins master), it’ll lie to you that the push was successful and the ref was moved, to make your client happy, but it actually hasn’t done so. Instead, it’ll just remember the ref that you wanted to push to (master in this case), and the actual commit you pushed.

Figure 11.3. Repository model

tres repos

Jenkins then schedules a build for your newly pushed commit. In this build, it’ll check out the ref from the upstream you wanted to push the changes into (origin/master in this case), then merge your changes into it. The build will then run as usual, and if the build was successful, this result of the merge that was just built and tested will be pushed to the upstream repository, becoming the new head of the branch. This process happens completely automatically.

Unlike Gerrit+Jenkins integration, in which each commit is separately built and tested, the validated merge in Jenkins only checks the tip of the ref. So you can split your change into several logically separated pieces, without paying the penalty of more build time.

The gate repository hosts additional tags that can be useful. For example, every submitted commit is accessible via changes/N where N is a build number, and every commit that was actually built is accessible via build/N (build/N is a result of a merge of the upstream and changes/N. This allows one developer to look into a failure found in a commit pushed by another developer and help him fix the problem.

Sending changes and have them validated

Let’s get back to the developer laptop and make a change to the code base. When you are done, make a commit. To see the effect of validation, we’ll have this change break the build.

# just to be clear that you are working on the master branch
$ git checkout master

# touch some file and commit
$ touch some.c
$ echo '#!/bin/sh\nfalse' > run.sh
$ git add some.c run.sh
$ git commit -am "adding some file"
[master 363f629] adding some file
 ...

Now the developer is happy with the changes. He thinks he’s done; he hasn’t run a full build/test cycle, but he thinks he made all the necessary changes for fixing a bug.

So let’s send this change to Jenkins to make sure they actually do pass all the tests.

$ git push jenkins master
 ...
remote: Created refs/heads/validated-merge-for/master/20120415-095424
remote:  at 363f629 for refs/heads/master
To ssh://myserver:2022/foo/repo.git
 * [new branch]      master -> master

Switch to the Jenkins web UI, and you’ll see a build scheduled for this new push. If you look at its console output, you’ll see that it’s checking out the upstream repo and then merging the change that was just submitted, and the result (8ded3bb) is getting built.

Using strategy: Build commits submitted for validated merge
Last Built Revision: Revision 4ced883 (master)
Fetching changes from 1 remote Git repository
Fetching upstream changes from /my/upstream.git
  ...
Merging refs/tags/changes/47
Commencing build of Revision 363f629 (master)
[workspace] $ /bin/sh -xe /tmp/hudson9087427762386246304.sh
+ ./run.sh
Build step 'Execute shell' marked build as failure
Finished: FAILURE

The build has failed, and so the push to the upstream repository didn’t happen. Let’s verify this from the perspective of Alice, another developer working on this code base. Pretend that you are Alice, and check out the workspace into another location. You’ll see that the master branch of Alice doesn’t contain the problematic 363f629.

To make this tutorial interesting, let’s make some changes as Alice in the master. In a big repository, it is normal for changes to overlap like this.

$ echo hello >> alice
$ git add hello
$ git commit -am "edit by alice"
[master 834782e] edit by alice
$ git push origin master
...
To /my/upstream.git
   458ea81..834782e  master -> master

At this point, the master branch in the upstream is 834782e as pushed by Alice, while our initial change 363f629 was rejected by Jenkins. We’ll see how this will resolve itself in the end.

Now, switch back to the original workspace, and let’s pretend that we found out that our initial commit was rejected because it didn’t pass the build. Now, make a correction to the commit, and push that to Jenkins again.

$ echo '#!/bin/sh\ntrue' > run.sh
$ git commit -am "fixed a broken build"
[master f664265] fixed a broken build
$ git push jenkins master
remote: Created refs/heads/validated-merge-for/master/20120415-105638
remote:   at f664265 for refs/heads/master
To ssh://myserver:2022/foo/repo.git
 * [new branch]      master -> master

When you make these corrections, you have the choice of amending the commit or doing a separate commit, and Jenkins can handle both correctly. In a silly regression like this, amending a commit is probably better, but for more intricate problems, leaving the record of a failure and its resolution could be useful.

Also note that we did not pull the changes Alice just made.

Now switch to the Jenkins UI once again, and see the console output of a newly scheduled build.

Using strategy: Build commits submitted for validated merge
Fetching changes from 1 remote Git repository
Fetching upstream changes from /my/upstream.git
  ...
Merging refs/tags/changes/48
Commencing build of Revision f884547 (master)
Checking out Revision f884547 (master)
[workspace] $ /bin/sh -xe /tmp/hudson1267891670488600118.sh
+ ./run.sh
Pushing the result to origin
Finished: SUCCESS

As you can see, at the very end Jenkins pushed the changes up to the upstream repository because the build was successful.

Go back to the workspace and pull from the origin to verify that the changes did make it into the upstream:

$ git pull origin
From /my/upstream
   458ea81..f884547  master     -> origin/master
Updating f664265..f884547
Fast-forward
 ...

This updates your local workspace by pulling in all the changes made in the trunk.

To more clearly see what has happened, run the git log to see the commit graph. We first committed 363f629, which didn’t work. Then Alice committed 834782e, which was pushed directly to the master. We then committed f664265 as a follow-up fix, which was merged with 834782e (then tip of the master branch) to produce f884547 (see the above console output for build #48 and you see that it actually tested this), which then became the tip of the master branch.

$ git log --decorate --graph --date-order
*   commit f884547 (HEAD, origin/master, master)
|\  Merge: 834782e f664265
| | Author: Jenkins
| |
| |     Merge commit 'refs/tags/changes/48'
| |
| * commit f664265 (jenkins/master)
| | Author: Kohsuke Kawaguchi
| |
| |     fixed a broken build
| |
* | commit 834782e
| | Author: Alice
| |
| |     edit by alice
| |
| * commit 363f629
|/  Author: Kohsuke Kawaguchi
|
|       adding some file
|
*   commit 458ea81

So now we completed a successful validated merge.