At work I found a fun problem: Building the current version of one of our services just failed under certain circumstances.
It turned out that happened because the build would succeed if there was a file somewhere in the build directory,
therefore making things seem to work when you did
mvn install and fail if you said
mvn clean install.
Time to find the version where that happened in git. There are probably a gazillion articles about git bisect, but I’ll just document what I did for myself.
As a first step I check out some old versions and try to find a good one that still compiles with
mvn clean install.
It is version 96eb424, and after trying I have checked it out, so I just tag it so I don’t need to remember the number
(I frequently use x- as a prefix for “temporary” tags, so I can easily clean them out, later):
mvn tag x-good-version
Ok, now back to the branch where I saw that problem, just the develop branch in this case:
git checkout develop mvn clean install
Yep, that still fails, I have correcly boxed in the version where the problem first occured. Let’s start the bisect:
git bisect start develop x-good-version
Now git checks out a version within the good to bad range, and I can run
mvn clean install
again. If this are OK with this version, I say
git bisect good, if the build fails I say
git bisect bad. By doing this
a few times over and over git will home in to the version that introduced the problem.
But wait, there’s more. I’m lazy.
So instead of saying all this, I let git do all the work. Maven is nice, because the maven command correctly sets the return
code to non-zero if it fails. So instead of me repeating
mvn clean install all over, I just let git do it:
mvn bisect run mvn clean install
And while hoping it would be busy long enough for me to fetch a cup of coffee, git quickly returned with the answer:
13d46f63700214c501a257d97c12378f9fe83147 is the first bad commit commit 13d46f63700214c501a257d97c12378f9fe83147
So now I can look what happened to 13d46f and fix that. And before I do so, I remember that number by
git tag x-bad 13d46f
and bring my repo to the normal state with
git bisect reset
And yes, at the end there’s the invisible elephant in the room, why didn’t Jenkins catch it? But that’s another tale.