Generic SCM plugin test cases

[Pekka Nikander]

[My apologies if this has been discussed earlier, but I didn't find in the archives.]

I'm working on the repo scm plugin, and needed to change the plugin-internal logic between calcRevisionsFromBuild and compareRemoteRevisionWith.  However, the exact relationship how they are supposed to work together does not seem to be that well documented... (and I don't know where to look at within the Jenkins source code).

So, I started to wonder if there are generic tests for SCM plugins, and if there are, how to a) run them and b) if they cover the "contract" between the calcRevisionsFromBuild and compareRemoteRevisionWith methods.

--Pekka Nikander

[Jesse Glick-4]

On 10/10/2012 03:56 AM, Pekka Nikander wrote:
> logic between calcRevisionsFromBuild and compareRemoteRevisionWith.  However, the exact
> relationship how they are supposed to work together does not seem to be that well documented

The Javadoc seems reasonably complete to me; anything specific missing?

> I don't know where to look at within the Jenkins source code

SCM.java itself, and implementations in various SCM plugins.

> if there are generic tests for SCM plugins

Not that I know of. That would be hard since a particular SCM plugin’s tests typically actually make or simulate SCM-specific changes in some test repository and then
monitor polling, changelog computation, checkouts, updates, and any other interesting operations.

[Pekka Nikander]

On Wednesday, 10 October 2012 15:57:24 UTC+3, Jesse Glick wrote:

On 10/10/2012 03:56 AM, Pekka Nikander wrote:
> logic between calcRevisionsFromBuild and compareRemoteRevisionWith.  However, the exact
> relationship how they are supposed to work together does not seem to be that well documented

The Javadoc seems reasonably complete to me; anything specific missing?

Maybe I'm asking for too much here (and should delve myself to the source), but reading the JavaDoc several times leaves me wondering how the two methods are really meant to work together.  Sure, I do understand that calcRFB is supposed to capture the revision state into an SCMRevisionState object, but what exactly needs to be captured there? Under what circumstances should compareRRW return INSIGNIFICANT vs. SIGNIFICANT?  

In a word, directly coming to this API through the plugin source code without any of the background does not lead me, even when going through the various other classes involved, into understanding what these APIs are really used for, i.e. what is the underlying polling mechanism that needs the information whether the repository has changed or not. Having a pointer to the right document / direction would be very helpful here; then, when I found the relevant wiki page on "Polling for changes", that is not really sufficient, and it also seems to be partially wrong.  

Anyway, in my particular case, what I am trying to do (and which more-or-less works already) is to enable using the Android repo tool with different configurations within a single project.  That is, I want to allow a Jenkins project to be parametrized so that the project can be used to build slightly different variants of Android.  I am then calling that project from a matrix project, using the parametrized trigger plugin, causing Jenkins to build the various variants of Android.  The way I have implemented that is to allow the repo SCM plugin to expand parameters that then determine what are the exact arguments the Android repo tool gets. Hence, in practise, depending on the parameters received from the parametrized trigger plugin, the repo plugin causes the Android repo tool to check out one or another branch, bug only for a few git repositories, keeping the 100+ other git repositories in the Android build identical.  For that I apparently need to record the EnvVars into the SCMRevisionState at calcRFB so that compareRRW is able to get the parameters, as it doesn't get any build object, only a project.

But to me this is somewhat weird, as I don't understand what Jenkins does with the compareRRW result.  In particular, is there any guarantee that if I return NONE or INSIGNIFICANT that the build Jenkins was considering would be really build that would use an equal EnvVars than that was recorded in the calcRFB or the previous computeRRW call?  What if it is considering another build with a different set of EnvVars, with the result that with those parameters repo would have changed the repo state?

 

> I don't know where to look at within the Jenkins source code

SCM.java itself, and implementations in various SCM plugins.

Thanks, I guess I need to go there.

 

> if there are generic tests for SCM plugins

Not that I know of. That would be hard since a particular SCM plugin’s tests typically actually make or simulate SCM-specific changes in some test repository and then monitor polling, changelog computation, checkouts, updates, and any other interesting operations.

Any SCM plugin having any good examples of those?  Right now the repo plugin has only very simple test cases, clearly not covering even a fraction of what really should be tested.

--Pekka

 

[Pekka Nikander]

> I don't know where to look at within the Jenkins source code

SCM.java itself, and implementations in various SCM plugins.

Thanks, I guess I need to go there.

Well, I now read SCM.java source code, and it doesn't help me much.  Basically, in SCM.java, the two method are only called from poll(), which is then called from outside of the class.  So, that doesn't help me to understand what are the supposed polling semantics, and under what conditions Jenkins does such polling, and why.

So, I would appreciate a pointer to the source code where the poll() or calcRevisionsFromBuild() and compareRemoteRevisionWith() are called from.

--Pekka Nikander

[Jesse Glick-4]

On 10/10/2012 02:48 PM, Pekka Nikander wrote:
> Under what circumstances should compareRRW return INSIGNIFICANT vs. SIGNIFICANT?

Generally SIGNIFICANT means that there were changes in sources and you want to do a new build as a result, whereas INSIGNIFICANT meant that there were changes but they
should have no effect on a build and should not trigger a new build—for example, changes in svn:ignore/.gitignore/.hgignore, or in files/modules that the user explicitly
indicated are to be ignored.

> that doesn't help me to understand what are the supposed polling semantics

Probably following code of an existing SCM implementation would help clarify things generally, though perhaps not in your case which seems rather specialized.

> the source code where the poll() or calcRevisionsFromBuild() and compareRemoteRevisionWith() are called from.

Find Usages from an IDE would give you this kind of information. At that point you will probably know more than me.

[Pekka Nikander]

> the source code where the poll() or calcRevisionsFromBuild() and compareRemoteRevisionWith() are called from.

Find Usages from an IDE would give you this kind of information. At that point you will probably know more than me.

Thanks Jesse, but call me old school or simply lazy, but I don't use an IDE, I still use emacs & command line.  (I wrote my first lines of code back in 1980.  I hand't even glimpsed on Jenkins plugin or core internals before this week.)

However, I did some digging into the source, mainly AbstractProject.java, and I think I have discovered a rather fundamental problem with polling implementation vs. parametrized builds.  In a nutshell, the current way polling is implemented is not compatible with parametrized SCM plugins, and more generally it may also result in builds not being made when they should, depending on how you user parameters in the project build scripts.  That is, if you have a parametrized project that is called from a build matrix through the parametrized trigger plugin, and if you do polling, the results are not well defined, at least as far as I can see. Fixing that seems to require a rather major rehaul of the polling mechanism, and I'm not inclined to do that.  (I probably shouldn't, given how little I know about Jenkins internals.  I'd be quite likely to screw things up.)

Anyway, for now, my plan is to simply implement a choice into the plugin: Either enable parameters, or polling.  If parameters are enabled, attempting to poll will result in some kind of an error, and if polling is enabled, attempting to use parameters in the plugin will result in another error.  Ugly, but saves me from being exposed to the gory details.

--Pekka