Jenkins gets killed / crashes unexpectedly

[Ove Ranheim]

Hi,

I experience this strange problem with Jenkins (build: #1468). It's running under its own non-privileged user on Centos 6. During execution of Maven 3 tests, all of a sudden the Jenkins process gets killed. I'm clueless to what's causing this.

The console log is really not too informative either:

INFO: Started SSHD at port 56903

Jun 12, 2012 10:37:40 AM jenkins.InitReactorRunner$1 onAttained

INFO: Completed initialization

Jun 12, 2012 10:37:43 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh

INFO: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@140c7272: display name [Root WebApplicationContext]; startup date [Tue Jun 12 10:37:43 CEST 2012]; root of context hierarchy

Jun 12, 2012 10:37:43 AM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory

INFO: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@140c7272]: org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429

Jun 12, 2012 10:37:43 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons

INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429: defining beans [filter,legacy]; root of factory hierarchy

Jun 12, 2012 10:37:43 AM hudson.TcpSlaveAgentListener <init>

INFO: JNLP slave agent listener started on TCP port 44944

Jun 12, 2012 10:37:56 AM hudson.WebAppMain$2 run

INFO: Jenkins is fully up and running

Jun 12, 2012 10:41:02 AM hudson.model.Run run

INFO: PoC Build #10 main build action completed: SUCCESS

Jun 12, 2012 11:06:03 AM hudson.triggers.SCMTrigger$Runner run

INFO: SCM changes detected in PoC Build. Triggering  #11

./j.sh: line 5: 25446 Killed                  java -jar jenkins.war --httpPort=8082 --prefix=/jenkins

Any idea to what's causing this?

-Ove

[Mark Waite]

When I've seen the message "Killed ..." it usually meant that there was an operating system process which caused the kill, either by kill <process_id> or by killall java, or something similar.

If that doesn't point to the source of the process death, you could use top to watch the memory size of the process, in case it is growing beyond the limits set by your operating system.

Mark Waite 

---

From: Ove Ranheim <[hidden email]>
To: [hidden email]
Sent: Tuesday, June 12, 2012 6:44 AM
Subject: Jenkins gets killed / crashes unexpectedly

Hi,

I experience this strange problem with Jenkins (build: #1468). It's running under its own non-privileged user on Centos 6. During execution of Maven 3 tests, all of a sudden the Jenkins process gets killed. I'm clueless to what's causing this.

The console log is really not too informative either:

INFO: Started SSHD at port 56903

Jun 12, 2012 10:37:40 AM jenkins.InitReactorRunner$1 onAttained

INFO: Completed initialization

Jun 12, 2012 10:37:43 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh

INFO: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@140c7272: display name [Root WebApplicationContext]; startup date [Tue Jun 12 10:37:43 CEST 2012]; root of context hierarchy

Jun 12, 2012 10:37:43 AM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory

INFO: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@140c7272]: org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429

Jun 12, 2012 10:37:43 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons

INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429: defining beans [filter,legacy]; root of factory hierarchy

Jun 12, 2012 10:37:43 AM hudson.TcpSlaveAgentListener <init>

INFO: JNLP slave agent listener started on TCP port 44944

Jun 12, 2012 10:37:56 AM hudson.WebAppMain$2 run

INFO: Jenkins is fully up and running

Jun 12, 2012 10:41:02 AM hudson.model.Run run

INFO: PoC Build #10 main build action completed: SUCCESS

Jun 12, 2012 11:06:03 AM hudson.triggers.SCMTrigger$Runner run

INFO: SCM changes detected in PoC Build. Triggering  #11

./j.sh: line 5: 25446 Killed                  java -jar jenkins.war --httpPort=8082 --prefix=/jenkins

Any idea to what's causing this?

-Ove

[Sami Tikka]

Try 1.469.

-- Sami

Ove Ranheim kirjoitti 12.6.2012 kello 15.44:

> Hi,
>
> I experience this strange problem with Jenkins (build: #1468). It's running under its own non-privileged user on Centos 6. During execution of Maven 3 tests, all of a sudden the Jenkins process gets killed. I'm clueless to what's causing this.
>
> The console log is really not too informative either:
>
> INFO: Started SSHD at port 56903
> Jun 12, 2012 10:37:40 AM jenkins.InitReactorRunner$1 onAttained
> INFO: Completed initialization
> Jun 12, 2012 10:37:43 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
> INFO: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@140c7272: display name [Root WebApplicationContext]; startup date [Tue Jun 12 10:37:43 CEST 2012]; root of context hierarchy
> Jun 12, 2012 10:37:43 AM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
> INFO: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@140c7272]: org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429
> Jun 12, 2012 10:37:43 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
> INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429: defining beans [filter,legacy]; root of factory hierarchy
> Jun 12, 2012 10:37:43 AM hudson.TcpSlaveAgentListener <init>
> INFO: JNLP slave agent listener started on TCP port 44944
> Jun 12, 2012 10:37:56 AM hudson.WebAppMain$2 run
> INFO: Jenkins is fully up and running
> Jun 12, 2012 10:41:02 AM hudson.model.Run run
> INFO: PoC Build #10 main build action completed: SUCCESS
> Jun 12, 2012 11:06:03 AM hudson.triggers.SCMTrigger$Runner run
> INFO: SCM changes detected in PoC Build. Triggering  #11
> ./j.sh: line 5: 25446 Killed                  java -jar jenkins.war --httpPort=8082 --prefix=/jenkins
>
> Any idea to what's causing this?
>
> -Ove
>

[Richard Mortimer]

It could be the kernel running out of virtual memory and the OOM (out of
memory) killer is killing jenkins. Run the "dmesg" command in a terminal
window on your jenkins server and see if the OOM kill has been active.
Also see.

https://wiki.jenkins-ci.org/display/JENKINS/I%27m+getting+OutOfMemoryError

Regards

Richard

On 12/06/2012 13:44, Ove Ranheim wrote:

> Hi,
>
> I experience this strange problem with Jenkins (build: #1468). It's
> running under its own non-privileged user on Centos 6. During execution
> of Maven 3 tests, all of a sudden the Jenkins process gets killed. I'm
> clueless to what's causing this.
>
> The console log is really not too informative either:
>
> INFO: Started SSHD at port 56903
> Jun 12, 2012 10:37:40 AM jenkins.InitReactorRunner$1 onAttained
> INFO: Completed initialization
> Jun 12, 2012 10:37:43 AM
> org.springframework.context.support.AbstractApplicationContext
> prepareRefresh
> INFO: Refreshing
> org.springframework.web.context.support.StaticWebApplicationContext@140c7272:
> display name [Root WebApplicationContext]; startup date [Tue Jun 12
> 10:37:43 CEST 2012]; root of context hierarchy
> Jun 12, 2012 10:37:43 AM
> org.springframework.context.support.AbstractApplicationContext
> obtainFreshBeanFactory
> INFO: Bean factory for application context
> [org.springframework.web.context.support.StaticWebApplicationContext@140c7272]:
> org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429
> Jun 12, 2012 10:37:43 AM
> org.springframework.beans.factory.support.DefaultListableBeanFactory
> preInstantiateSingletons
> INFO: Pre-instantiating singletons in
> org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429:
> defining beans [filter,legacy]; root of factory hierarchy
> Jun 12, 2012 10:37:43 AM hudson.TcpSlaveAgentListener <init>
> INFO: JNLP slave agent listener started on TCP port 44944
> Jun 12, 2012 10:37:56 AM hudson.WebAppMain$2 run
> INFO: Jenkins is fully up and running
> Jun 12, 2012 10:41:02 AM hudson.model.Run run
> INFO: PoC Build #10 main build action completed: SUCCESS
> Jun 12, 2012 11:06:03 AM hudson.triggers.SCMTrigger$Runner run
> INFO: SCM changes detected in PoC Build. Triggering #11
> ./j.sh: line 5: *25446 Killed java -jar jenkins.war --httpPort=8082
> --prefix=/jenkins*
>
> Any idea to what's causing this?
>
> -Ove
>

[Ove Ranheim]

Thanks Richard,

This was helpful. The reason to this problem was two things: I used OpenJDK 1.6 on Centos-6 and heap/mem had to be changed for both Jenkins instance and Maven build.

After switching to Oracle 1.6 JDK it works fine.

Ove

On Jun 13, 2012, at 1:09 AM, Richard Mortimer wrote:

> It could be the kernel running out of virtual memory and the OOM (out of memory) killer is killing jenkins. Run the "dmesg" command in a terminal window on your jenkins server and see if the OOM kill has been active. Also see.
>
> https://wiki.jenkins-ci.org/display/JENKINS/I%27m+getting+OutOfMemoryError
>
> Regards
>
> Richard
>
> On 12/06/2012 13:44, Ove Ranheim wrote:
>> Hi,
>>
>> I experience this strange problem with Jenkins (build: #1468). It's
>> running under its own non-privileged user on Centos 6. During execution
>> of Maven 3 tests, all of a sudden the Jenkins process gets killed. I'm
>> clueless to what's causing this.
>>
>> The console log is really not too informative either:
>>
>> INFO: Started SSHD at port 56903
>> Jun 12, 2012 10:37:40 AM jenkins.InitReactorRunner$1 onAttained
>> INFO: Completed initialization
>> Jun 12, 2012 10:37:43 AM
>> org.springframework.context.support.AbstractApplicationContext
>> prepareRefresh
>> INFO: Refreshing
>> org.springframework.web.context.support.StaticWebApplicationContext@140c7272:
>> display name [Root WebApplicationContext]; startup date [Tue Jun 12
>> 10:37:43 CEST 2012]; root of context hierarchy
>> Jun 12, 2012 10:37:43 AM
>> org.springframework.context.support.AbstractApplicationContext
>> obtainFreshBeanFactory
>> INFO: Bean factory for application context
>> [org.springframework.web.context.support.StaticWebApplicationContext@140c7272]:
>> org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429
>> Jun 12, 2012 10:37:43 AM
>> org.springframework.beans.factory.support.DefaultListableBeanFactory
>> preInstantiateSingletons
>> INFO: Pre-instantiating singletons in
>> org.springframework.beans.factory.support.DefaultListableBeanFactory@3b896429:
>> defining beans [filter,legacy]; root of factory hierarchy
>> Jun 12, 2012 10:37:43 AM hudson.TcpSlaveAgentListener <init>
>> INFO: JNLP slave agent listener started on TCP port 44944
>> Jun 12, 2012 10:37:56 AM hudson.WebAppMain$2 run
>> INFO: Jenkins is fully up and running
>> Jun 12, 2012 10:41:02 AM hudson.model.Run run
>> INFO: PoC Build #10 main build action completed: SUCCESS
>> Jun 12, 2012 11:06:03 AM hudson.triggers.SCMTrigger$Runner run
>> INFO: SCM changes detected in PoC Build. Triggering #11
>> ./j.sh: line 5: *25446 Killed java -jar jenkins.war --httpPort=8082
>> --prefix=/jenkins*
>>
>> Any idea to what's causing this?
>>
>> -Ove
>>