[JIRA] (JENKINS-15066) Long wait between last build step and first post build step

Markus Strand created JENKINS-15066

Long wait between last build step and first post build step

Issue Type:	Bug
Assignee:	Unassigned
Components:	core
Created:	07/Sep/12 7:46 AM
Description:	Were are struggling with our builds that are occasionally jamming in between the last build step and first post-build action. The more builds we are running in parallel the more likely we'll face this issue. We are running Jenkins 1.454 and "execute concurrent builds" option to run multiple instances of the same job. Our master is Windows server 2008 and so are the slaves. All the builds are happening on the slaves, master is just dispatching the jobs. Slaves are virtual machines. We are not running out of memory. Any help is appreciated. For example what log recorders we could use to debug this issue? Jenkins default log doesn't provide any further info and neither does the console log.
Project:	Jenkins
Priority:	Major
Reporter:	Markus Strand

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

kutzi commented on JENKINS-15066

Long wait between last build step and first post build step

Try to get some thread dumps of the 'jammed' builds (kill -3 or jstack) and post them here!

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Markus Strand updated JENKINS-15066

Long wait between last build step and first post build step

jstack from a jammed build
Change By:	Markus Strand (17/Sep/12 6:56 AM)
Attachment:	jstack_jammed.log

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

 

Markus Strand edited a comment on JENKINS-15066

Long wait between last build step and first post build step

jstack from a jammed build It is the sawnwp110_image1 slave that is running the jammed job.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

kutzi commented on JENKINS-15066

Long wait between last build step and first post build step

This looks like the thread dump of the master process. We also need the thread dump of the VM process on the slave which actually runs the build.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Markus Strand updated JENKINS-15066

Long wait between last build step and first post build step

It seems like the template project plugin is causing this behaviour when running builds in parallel. The https://github.com/jenkinsci/template-project-plugin/blob/master/src/main/java/hudson/plugins/templateproject/ProxyPublisher.java#L43 """ return BuildStepMonitor.STEP; """ Will cause the Method perform to block in http://sorcerer.jenkins-ci.org/source-view.html?hudson/tasks/BuildStepMonitor.js#34
Change By:	Markus Strand (18/Sep/12 11:18 AM)
Component/s:	template-project

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

kutzi updated JENKINS-15066

Long wait between last build step and first post build step

Change By:	kutzi (20/Jan/13 3:59 PM)
Component/s:	core

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira