Is it possible to block a build job if another one is running which is not part of up- or downstream?

[Jan Seidel-2]

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

Cheers

Jan

[wohauser-3]

Hi Jan,

 

have a little look at the "locks-and-latches" plugin.

 

This should provide the lock you need.

e.g. set up the lock "database" in jenkins configuration

use the lock in all database related jobs.

 

Regards

Wolfgang

 

[Matthew.Webber]

There are several plugins which might be worth looking at.

The “locks and latches” plugin has already been mentioned, but this is listed for possible deprecation (see <a href="https://wiki.jenkins-ci.org/display/JENKINS/Proposed&#43;Plugin&#43;Deprecation"> https://wiki.jenkins-ci.org/display/JENKINS/Proposed+Plugin+Deprecation). You might want to look at the Exclusion plugin, which we use.

Matthew

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Jan Seidel
Sent: 28 June 2012 11:41
To: [hidden email]
Subject: Is it possible to block a build job if another one is running which is not part of up- or downstream?

 

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

 

 

Cheers

Jan

[Bap]

Quoting Jan Seidel <[hidden email]>:

> Hi folks,
>
> I am trying to parallelize some of our builds to speed things up.
> This particular build is quite special as it also interacts with databases.
> Multiple write access on a database will wreck the content, so this must be
> avoided by all means. It takes us in worst cast out of business for 2 weeks
> and creates loads of work and stress.
>
> Don't ask me why the DB design is as it is, that's pretty complicated,
> insane, sooo wrong and not worth discussing in order to fix that issue and
> let the build jobs just do what they are meant to do.
> Have been there, didn't like it, went away!
>
> The initial build job is a dispatcher that decides which job to run. So far
> quite easy but it also checks for 2 conditions which requires to almost
> identical jobs asides of one of the repositories location.
> *So the jobs access mostly the same ressources including the database which
> must not be simultaneously!* So they are a kind of "*siblings*"
> The default is set that inhibits to spawn duplicate jobs but with this
> second condition is it changing a bit. The second condition requires to
> block a job if a duplicate or the sibling is running
>
> This brings me to a dumb situation. Either I find a way to:
>
>    - refer to related AND non-related jobs which may block a build
>    - block on top-level (the dispatcher). This works but scraps all my
>    efforts to get jobs not sharing same ressources to run in parallel
>    - do not block jobs and hope that the developers don't botch it and
>    wreck a database *yuk*
>
> Do you know a solution to block a job if a clone or a sibling already is
> running?
>

https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin

Create your "lock" in the global config:
category = "db-modifiers"
max total concurrent builds = 1

Then enable throttle concurrent builds in any job that modifies the  
database, and select the category.


HTH,
Bap.

>
>
> Cheers
>
> Jan

[Sami Tikka]

I'll add another way to exclude other jobs: make a slave with only one executor and tie the jobs to restrict to the slave. Only one at a time will run. 

-- Sami

Jan Seidel <[hidden email]> kirjoitti 28.6.2012 kello 13.41:

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

Cheers

Jan

[Jan Seidel-2]

Hi folks,

thanks for your replies :)

@Sami: that's actually the way the jobs are handled at the moment but that's not supporting the idea to parallelize jobs ;)

• I will take a closer looks at locks and latches. I did actually use it some time ago but I'm not sure if it will work the way I need it to work as the "weight" does just dictate which job is superior but not exclusive.
• The exclusive execution plugin is not of help as it blocks all follow up jobs until it has been excuted....on all nodes. So even non-related jobs won't run for that time. That would create a tremendous and not copable backlog.
• Throttle concurrent builds plugin is a nice one but also not matching.

My problem is that I need a possibility to define myself which job is concurrent.
Maybe I should elaborate my issue a bit more.

Our software is produced in several stages.
Two of these stages are the major ones.

• First stage is generating java code via GUIDE. That to is a visual editor to create a GUI and its logic, and then generate the appropriate java code from the project which is incorporated into the manually written code.
• Second stage is to build the software bundle from the entire code.

We have currently a developer and a delivery branch for each software bundle. This way we keep our code freeze for building software down to round about 4-6 minutes while a snapshot from the developer branch is copied to the delivery branch.

Everything works fine so far BUT if we have to provide a bugfix are troubles ahead.

• The auto-generated code from GUIDE should then be stored in the delivery branch and not the developer branch.
• The SCM module in Jenkins needs hardcoded repository paths. Dynamic assembly of the path with variables will fail. So the job is duplicated only with 2 different repository paths which works well so far.
• The problem I am fighting with is that the developers work on the developer branch and every now and then also start GUIDE to integrate/fix/validate new implementations. This must not happen in parallel with the build job that generates into the delivery branch but will do it in case of a bugfixing as Jenkins assumes these jobs as non-related. How should it know that a build step accessing the database has to be exclusive.
• Non-exclusive access may break the database which is no good at all as the database then has to be fixed manually. And that's no joy with several thousand entries per run.

The most simple way would be if a plugin exists where you just enter other projects and then could select block this or the other jobs if one of the defined projects is running but I haven't seen anything like that yet.

Take care

Jan

Am Donnerstag, 28. Juni 2012 21:08:06 UTC+2 schrieb sti:

I'll add another way to exclude other jobs: make a slave with only one executor and tie the jobs to restrict to the slave. Only one at a time will run. 

-- Sami

Jan Seidel <[hidden email]> kirjoitti 28.6.2012 kello 13.41:

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

Cheers

Jan

[Sami Tikka]

Well, the idea was to tie only those jobs that need exclusive access to the database to the slave with 1 executor. The rest of the jobs can be left to run where ever. You can add more slaves to parallelize the build as much as you like and use the slave with 1 executor as the "choke point" for database access.

-- Sami

Jan Seidel kirjoitti 3.7.2012 kello 9.33:

> Hi folks,
>
> thanks for your replies :)
>
> @Sami: that's actually the way the jobs are handled at the moment but that's not supporting the idea to parallelize jobs ;)
>
> • I will take a closer looks at locks and latches. I did actually use it some time ago but I'm not sure if it will work the way I need it to work as the "weight" does just dictate which job is superior but not exclusive.
> • The exclusive execution plugin is not of help as it blocks all follow up jobs until it has been excuted....on all nodes. So even non-related jobs won't run for that time. That would create a tremendous and not copable backlog.
> • Throttle concurrent builds plugin is a nice one but also not matching.
> My problem is that I need a possibility to define myself which job is concurrent.
> Maybe I should elaborate my issue a bit more.
>
> Our software is produced in several stages.
> Two of these stages are the major ones.
> • First stage is generating java code via GUIDE. That to is a visual editor to create a GUI and its logic, and then generate the appropriate java code from the project which is incorporated into the manually written code.
> • Second stage is to build the software bundle from the entire code.
> We have currently a developer and a delivery branch for each software bundle. This way we keep our code freeze for building software down to round about 4-6 minutes while a snapshot from the developer branch is copied to the delivery branch.
>
> Everything works fine so far BUT if we have to provide a bugfix are troubles ahead.
> • The auto-generated code from GUIDE should then be stored in the delivery branch and not the developer branch.
> • The SCM module in Jenkins needs hardcoded repository paths. Dynamic assembly of the path with variables will fail. So the job is duplicated only with 2 different repository paths which works well so far.
> • The problem I am fighting with is that the developers work on the developer branch and every now and then also start GUIDE to integrate/fix/validate new implementations. This must not happen in parallel with the build job that generates into the delivery branch but will do it in case of a bugfixing as Jenkins assumes these jobs as non-related. How should it know that a build step accessing the database has to be exclusive.
> • Non-exclusive access may break the database which is no good at all as the database then has to be fixed manually. And that's no joy with several thousand entries per run.
> The most simple way would be if a plugin exists where you just enter other projects and then could select block this or the other jobs if one of the defined projects is running but I haven't seen anything like that yet.
>
>
>
> Take care
>
> Jan
>
>
> Am Donnerstag, 28. Juni 2012 21:08:06 UTC+2 schrieb sti:
> I'll add another way to exclude other jobs: make a slave with only one executor and tie the jobs to restrict to the slave. Only one at a time will run.
>
> -- Sami
>
> Jan Seidel <[hidden email]> kirjoitti 28.6.2012 kello 13.41:
>
>> Hi folks,
>>
>> I am trying to parallelize some of our builds to speed things up.
>> This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.
>>
>> Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
>> Have been there, didn't like it, went away!
>>
>> The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
>> So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
>> The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running
>>
>> This brings me to a dumb situation. Either I find a way to:
>> • refer to related AND non-related jobs which may block a build
>> • block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
>> • do not block jobs and hope that the developers don't botch it and wreck a database *yuk*
>> Do you know a solution to block a job if a clone or a sibling already is running?
>>
>>
>>
>>
>>
>> Cheers
>>
>> Jan
>>

[Jan Seidel-2]

Ah! now I'm getting it :)
yesnomaybe... The idea is not bad but I have to investigate if it is feasible.

• The workspaces for these particular jobs are not small and have to share harddisk space with other jobs. Currently does one machine build all these special jobs as it for this purpose has several TB of disk space. This could require some hardware reconfiguration. No fun but possible.
• Worse is that I would have to adapt all boxes to meet the requirements for these jobs. The environment is quite special and tricky. It is already a hassle set it newly up on that single machine but to recreate on productive machines which already are set up needs some special treatment to ensure that nothing is interferring.
• The machines provide per box 3 nodes with up to 3 executors per node so that would not be a problem to make a split for the exclusive jobs.

I have to analyze it but this is probably the smartest approach if no convenient plugin is available.

Cheers
Jan

Am Donnerstag, 28. Juni 2012 12:41:04 UTC+2 schrieb Jan Seidel:

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

Cheers

Jan

[Sami Tikka]

A slave does not have to run in separate hardware. You can run one or more slaves in the same server where Jenkins master is running. The slaves just need to have a unique name and working directory.

By the way, if you have jobs reaching into other jobs' workspaces, you might run into difficulties when you try to parallelize the build pipeline. Jenkins does not make any provisions for synchronizing access to a job workspace, so if that's what you are doing, you have to take care of it yourself.

The Jenkins way is to archive the build artifacts (which stores them into a special area under JENKINS_HOME, outside of any job workspace) and then use Copy Artifacts build step to copy the needed artifacts for a job that needs them. Of course, if your build artifacts are in the gigabytes range, copying them around will take some time. Fast SSD might help, or you might choose to dip into the workspaces directly, but be aware of the synchronization issue.

-- Sami

Jan Seidel kirjoitti 3.7.2012 kello 11.37:

> Ah! now I'm getting it :)
> yesnomaybe... The idea is not bad but I have to investigate if it is feasible.
>
> • The workspaces for these particular jobs are not small and have to share harddisk space with other jobs. Currently does one machine build all these special jobs as it for this purpose has several TB of disk space. This could require some hardware reconfiguration. No fun but possible.
> • Worse is that I would have to adapt all boxes to meet the requirements for these jobs. The environment is quite special and tricky. It is already a hassle set it newly up on that single machine but to recreate on productive machines which already are set up needs some special treatment to ensure that nothing is interferring.
> • The machines provide per box 3 nodes with up to 3 executors per node so that would not be a problem to make a split for the exclusive jobs.
> I have to analyze it but this is probably the smartest approach if no convenient plugin is available.
>
> Cheers
> Jan
>
> Am Donnerstag, 28. Juni 2012 12:41:04 UTC+2 schrieb Jan Seidel:
> Hi folks,
>
> I am trying to parallelize some of our builds to speed things up.
> This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.
>
> Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
> Have been there, didn't like it, went away!
>
> The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
> So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
> The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running
>
> This brings me to a dumb situation. Either I find a way to:
> • refer to related AND non-related jobs which may block a build
> • block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
> • do not block jobs and hope that the developers don't botch it and wreck a database *yuk*
> Do you know a solution to block a job if a clone or a sibling already is running?
>
>
>
>
>
> Cheers
>
> Jan
>

[Jan Seidel-2]

I think the two of us are getting out of sync ;)

• We use 4 machines: 1 runs the master and 2 slaves and all other machines have 3 slaves with up to 3 executors running.
• The jobs workspaces are not reaching into each others. I have tried the sync issue (been there, didn't like it, went away)
• What I meant in the last post: The problem is rather on OS level as some of the programs and tools don't like each other. So installing both categories may lead to crashes or other less obvious problems. That's why the 2 categories of jobs have been separated physically on cost of performance.
• Our machines for building the final software run on SSD but that won't work for GUIDE as the array would have to be big and thereby expensive. And SSD is unfortunately not as solid as expected. I did wreck the first SSD within 6 weeks due to the massive disk activities caused by the continous integration :)

Looks like I have to consider that my next big project would be to review our toolbox to get things less sophisticated and more compatible. It's time to clean up the mess that did grow over the years before I came to business.
I think I will use your suggestion at time being and setup single 1-executor nodes with exclusive job assignments until I have finish my migrations and then return to the issue when I have less workload. This is the easiest and quickest solution.

Thanks :)
Jan



Am Dienstag, 3. Juli 2012 12:16:30 UTC+2 schrieb sti:

A slave does not have to run in separate hardware. You can run one or more slaves in the same server where Jenkins master is running. The slaves just need to have a unique name and working directory.

By the way, if you have jobs reaching into other jobs' workspaces, you might run into difficulties when you try to parallelize the build pipeline. Jenkins does not make any provisions for synchronizing access to a job workspace, so if that's what you are doing, you have to take care of it yourself.

The Jenkins way is to archive the build artifacts (which stores them into a special area under JENKINS_HOME, outside of any job workspace) and then use Copy Artifacts build step to copy the needed artifacts for a job that needs them. Of course, if your build artifacts are in the gigabytes range, copying them around will take some time. Fast SSD might help, or you might choose to dip into the workspaces directly, but be aware of the synchronization issue.

-- Sami

Jan Seidel kirjoitti 3.7.2012 kello 11.37:

> Ah! now I'm getting it :)
> yesnomaybe... The idea is not bad but I have to investigate if it is feasible.
>
>         • The workspaces for these particular jobs are not small and have to share harddisk space with other jobs. Currently does one machine build all these special jobs as it for this purpose has several TB of disk space. This could require some hardware reconfiguration. No fun but possible.
>         • Worse is that I would have to adapt all boxes to meet the requirements for these jobs. The environment is quite special and tricky. It is already a hassle set it newly up on that single machine but to recreate on productive machines which already are set up needs some special treatment to ensure that nothing is interferring.
>         • The machines provide per box 3 nodes with up to 3 executors per node so that would not be a problem to make a split for the exclusive jobs.
> I have to analyze it but this is probably the smartest approach if no convenient plugin is available.
>
> Cheers
> Jan
>
> Am Donnerstag, 28. Juni 2012 12:41:04 UTC+2 schrieb Jan Seidel:
> Hi folks,
>
> I am trying to parallelize some of our builds to speed things up.
> This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.
>
> Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
> Have been there, didn't like it, went away!
>
> The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
> So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
> The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running
>
> This brings me to a dumb situation. Either I find a way to:
>         • refer to related AND non-related jobs which may block a build
>         • block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
>         • do not block jobs and hope that the developers don't botch it and wreck a database *yuk*
> Do you know a solution to block a job if a clone or a sibling already is running?
>
>
>
>
>
> Cheers
>
> Jan
>

[Lynn Lin-2]

Lynn

在 2012-6-28 下午7:32，"Bap" <[hidden email]>写道：

Quoting Jan Seidel <[hidden email]>:

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases.
Multiple write access on a database will wreck the content, so this must be
avoided by all means. It takes us in worst cast out of business for 2 weeks
and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated,
insane, sooo wrong and not worth discussing in order to fix that issue and
let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far
quite easy but it also checks for 2 conditions which requires to almost
identical jobs asides of one of the repositories location.
*So the jobs access mostly the same ressources including the database which
must not be simultaneously!* So they are a kind of "*siblings*"
The default is set that inhibits to spawn duplicate jobs but with this
second condition is it changing a bit. The second condition requires to
block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

   - refer to related AND non-related jobs which may block a build
   - block on top-level (the dispatcher). This works but scraps all my
   efforts to get jobs not sharing same ressources to run in parallel
   - do not block jobs and hope that the developers don't botch it and
   wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is
running?

https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin

Create your "lock" in the global config:
category = "db-modifiers"
max total concurrent builds = 1

Then enable throttle concurrent builds in any job that modifies the database, and select the category.


HTH,
Bap.

Cheers

Jan

[Lynn Lin-2]

Lynn

在 2012-6-28 下午6:41，"Jan Seidel" <[hidden email]>写道：

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

Cheers

Jan

[Jan Seidel-2]

Hi Lynn,

unfortunately your qoute is not displayed in original.
But hitting the translation button shows a whacko translation that in its wicked way makes sense XD
I will check it out.

Tanks
Jan

Am Donnerstag, 28. Juni 2012 12:41:04 UTC+2 schrieb Jan Seidel:

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

Cheers

Jan

[Jan Seidel-2]

Hi Lynn,

this looks promising so far.
I cannot test it right now as my locks&latches seems smacked up. some entries are missing.
A category can be created and also assigned to a job but the rest like: number of concurrent builds, number of executors, multi project and even single project are missing so I cannot activate it right now.
Hopefully an update of the system will fix it.

I'll keep you posted
Jan

Am Donnerstag, 28. Juni 2012 12:41:04 UTC+2 schrieb Jan Seidel:

Hi folks,

I am trying to parallelize some of our builds to speed things up.
This particular build is quite special as it also interacts with databases. Multiple write access on a database will wreck the content, so this must be avoided by all means. It takes us in worst cast out of business for 2 weeks and creates loads of work and stress.

Don't ask me why the DB design is as it is, that's pretty complicated, insane, sooo wrong and not worth discussing in order to fix that issue and let the build jobs just do what they are meant to do.
Have been there, didn't like it, went away!

The initial build job is a dispatcher that decides which job to run. So far quite easy but it also checks for 2 conditions which requires to almost identical jobs asides of one of the repositories location.
So the jobs access mostly the same ressources including the database which must not be simultaneously! So they are a kind of "siblings"
The default is set that inhibits to spawn duplicate jobs but with this second condition is it changing a bit. The second condition requires to block a job if a duplicate or the sibling is running

This brings me to a dumb situation. Either I find a way to:

• refer to related AND non-related jobs which may block a build
• block on top-level (the dispatcher). This works but scraps all my efforts to get jobs not sharing same ressources to run in parallel
• do not block jobs and hope that the developers don't botch it and wreck a database *yuk*

Do you know a solution to block a job if a clone or a sibling already is running?

Cheers

Jan