Quantcast

Jenkins slow coming up and clicking on views with builds with lots of history

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Jenkins slow coming up and clicking on views with builds with lots of history

swerner
We recently moved off Hudson to the latest Jenkins (1.475).

After doing so, we have noticed an increased startup time and clicking on tabs takes longer too. Upon further investigation, these jobs typically have a history of 100 or so builds.

Dumping the stack while waiting produces the following:
   java.lang.Thread.State: RUNNABLE
        at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
        at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
        at java.io.File.exists(File.java:733)
        at hudson.model.AbstractBuild.calcChangeSet(AbstractBuild.java:828)
        at hudson.model.AbstractBuild.getChangeSet(AbstractBuild.java:806)
        at hudson.model.View$People.isApplicable(View.java:674)
        at hudson.model.View.hasPeople(View.java:596)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
        at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
        at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
        at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
        at org.apache.commons.jexl.parser.ASTOrNode.value(ASTOrNode.java:61)
        at org.apache.commons.jexl.parser.ASTExpression.value(ASTExpression.java:54)
        at org.apache.commons.jexl.parser.ASTExpressionExpression.value(ASTExpressionExpression.java:56)
        at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
        at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
...

To complicate things more, we actually run on a clustered filesystem (gluster), this we cannot change. We have noticed internally checking the existence of files and listing is significantly slower than a local filesystem, but the redundancy outweighs the slight performance impact. Before the upgrade, our Hudson was working fine on gluster. There was a slight boot delay, but never saw the lag on clicking on tabs like Jenkins. Did something change between distributions that you need to access the history everytime you view, can you not cache it? Also, most of our builds invoke shell scripts or call other java programs, there are no scm builds, so the calcChangeSet() is always empty, is there a way of bypassing the need of this call when scm is set to None?

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

RE: Jenkins slow coming up and clicking on views with builds with lots of history

Matthew.Webber

Starting Jenkins with -Dhudson.model.Hudson.logStartupPerformance=true might give you some additional information about startup time. See <a href="https://wiki.jenkins-ci.org/display/JENKINS/Features&#43;controlled&#43;by&#43;system&#43;properties"> https://wiki.jenkins-ci.org/display/JENKINS/Features+controlled+by+system+properties

 

We run Jenkins off a local disk for speed, and back up Jenkins home once a day (excluding the job history, which if we lost it we could live without).

 

Matthew

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of swerner
Sent: 27 July 2012 15:26
To: [hidden email]
Subject: Jenkins slow coming up and clicking on views with builds with lots of history

 

We recently moved off Hudson to the latest Jenkins (1.475).

After doing so, we have noticed an increased startup time and clicking on tabs takes longer too. Upon further investigation, these jobs typically have a history of 100 or so builds.

Dumping the stack while waiting produces the following:
   java.lang.Thread.State: RUNNABLE
        at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
        at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
        at java.io.File.exists(File.java:733)
        at hudson.model.AbstractBuild.calcChangeSet(AbstractBuild.java:828)
        at hudson.model.AbstractBuild.getChangeSet(AbstractBuild.java:806)
        at hudson.model.View$People.isApplicable(View.java:674)
        at hudson.model.View.hasPeople(View.java:596)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
        at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
        at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
        at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
        at org.apache.commons.jexl.parser.ASTOrNode.value(ASTOrNode.java:61)
        at org.apache.commons.jexl.parser.ASTExpression.value(ASTExpression.java:54)
        at org.apache.commons.jexl.parser.ASTExpressionExpression.value(ASTExpressionExpression.java:56)
        at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
        at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
...

To complicate things more, we actually run on a clustered filesystem (gluster), this we cannot change. We have noticed internally checking the existence of files and listing is significantly slower than a local filesystem, but the redundancy outweighs the slight performance impact. Before the upgrade, our Hudson was working fine on gluster. There was a slight boot delay, but never saw the lag on clicking on tabs like Jenkins. Did something change between distributions that you need to access the history everytime you view, can you not cache it? Also, most of our builds invoke shell scripts or call other java programs, there are no scm builds, so the calcChangeSet() is always empty, is there a way of bypassing the need of this call when scm is set to None?

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

RE: Jenkins slow coming up and clicking on views with builds with lots of history

nathang
We had similar issues regardless of used version. We tracked them to fingerprint analysis (though we dont use fingerprints explicitly anywhere) and subsequently the copyartifacts plugin from version greater than 1.19 i think. If you dont use fingerprints, just delete their folder to see if it'll help. If it does, downgrade copyartifacts to prevent redundant fingerprints accumulation.
Loading...