Jenkins › Jenkins issues

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.441

Classic

List

Threaded

34 messages Options

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.441

mark streit created JENKINS-12037:
-------------------------------------

Summary: CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.441
Key: JENKINS-12037
URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
Project: Jenkins
Issue Type: Bug
Components: cli
Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
* Tomcat 7.0.22
* JDK 1.6.0_14
* Only ONE Master configuration - no slaves are configured
* 3 Executors - (one less than the max number of CPUs)
Reporter: mark streit

We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
Log:
[FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar

Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.

We did NOT try to make any adjustments to the Tomcat configuration:

Tomcat Connector connectionUploadTimeout

but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.

================================================================================================

We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).

Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.

Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: kohsuke
Build-Jdk: 1.6.0_26
Main-Class: hudson.cli.CLI
Jenkins-CLI-Version: 1.441

Under Tomcat 7, we get this stacktrace:

Started by command line
[workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
+ /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Request.call(Request.java:149)
at hudson.remoting.Channel.call(Channel.java:681)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
at $Proxy2.main(Unknown Source)
at hudson.cli.CLI.execute(CLI.java:200)
at hudson.cli.CLI._main(CLI.java:330)
at hudson.cli.CLI.main(CLI.java:245)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Request.abort(Request.java:273)
at hudson.remoting.Channel.terminate(Channel.java:732)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
Build step 'Execute shell' marked build as failure
Notifying upstream projects of job completion
Finished: FAILURE

Under Winstone, we get this stacktrace - it's somewhat different:

Started by command line
[workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
+ /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
java.io.IOException: Premature EOF
at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
at hudson.remoting.Request.call(Request.java:149)
at hudson.remoting.Channel.call(Channel.java:681)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
at $Proxy2.main(Unknown Source)
at hudson.cli.CLI.execute(CLI.java:200)
at hudson.cli.CLI._main(CLI.java:330)
at hudson.cli.CLI.main(CLI.java:245)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
at hudson.remoting.Request.abort(Request.java:273)
at hudson.remoting.Channel.terminate(Channel.java:732)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
Caused by: java.io.IOException: Premature EOF
at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
Build step 'Execute shell' marked build as failure
Notifying upstream projects of job completion
Finished: FAILURE

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Remove Ads

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.441

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=156495#comment-156495 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

For completeness can you get the associated logs from the jenkins server side. That should confirm that the server side is closing the downstream socket because it has detected inactivity/timeout on the upstream side.

How long does the cli command take to fail? A rough (to the nearest second or so) timing is all that is needed. The issue was due to webserver connection timeout before so it is likely to be something like 60, 30, 20 or 15 seconds.

Are you able to get a packet trace of the network traffic between cli and jenkins? capturing from the cli side with wireshark or tcpdump should be fine. If you are able to get that and can share it then drop me an email at [hidden email] so that we can arrange to transfer it in private.

I don't have a lot of time at the moment so if anyone else wants to take a look instead of me please feel free to do so!

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.441
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.441

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=156497#comment-156497 ]

mark streit commented on JENKINS-12037:
---------------------------------------

I will get logs from the Tomcat server. It also failed with Winstone this time whereas before it worked with Winstone when using the 1.441 CLI. I will check on those logs assuming they are somewhere in the location where we put the WAR. Failure hit within 10 sec each time.

Thanks

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

mark streit updated JENKINS-12037:
----------------------------------

Summary: CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449 (was: CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.441)
Attachment: Tomcat7_Jenkins1449_logs.zip

Have attached logs from the Tomcat server where we are running this. We have downloaded the last version 1.449 WAR and the corresponding CLI jar for 1.449. We repeated the test and are seeing this I/O error in chunked connection again. (right now only 1.441 running in Winstone will work without this error).

Console output from the Jenkins web page is also included as is the Shell command text that occupies the Execute Shell section of the build job configuration page.

This all works with 1.441 as long as the container is Winstone.

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159184#comment-159184 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

I'm still max'd out with $dayjob and have not had time to look in much detail at this.

I just did a quick test on my test system with 1.451 using tomcat 6 and do not see any timeouts there with the tomcat http data upload timeout set to 20 seconds. That rules out any basic brokenness (I hope!).

Looking at your log files (see below) there is one instance where an error occurs 12 seconds after the previous timeout. That suggests that tomcat 7 is timing out in around 10 seconds. The jenkins channel keepalive ping is set for 15 seconds so it could be that the ping is still not aggressive enough.

We probably just need to test with a 5 seconds ping time and see if that fixes things.

First error reported in ConsoleOutput.log
{code}
Feb 6, 2012 2:47:38 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel Chunked connection to http://99.99.99.94:8082/jenkins/cli
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
{code}
This corresponds to the following two connections in localhost_access_log.2012-02-06.txt
{code}
10.25.50.94 - - [06/Feb/2012:14:47:38 -0500] "POST /jenkins/cli HTTP/1.1" 200 13935
10.25.50.94 - - [06/Feb/2012:14:47:38 -0500] "POST /jenkins/cli HTTP/1.1" 200 -
{code}
The former is the downstream link and the latter is the upstream (sending from cli to jenkins) hence there being no output in that direction.

catalina.out (equivalent to standalone jenkins.log) has the other side of the error
This confirms that the upstream reader sees the input data closed by tomcat.
{code}
Feb 6, 2012 2:47:38 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel HTTP full-duplex channel b625b58a-d9d7-4a42-856c-287e953edb47
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
{code}

The next error in the console output occurs 12 seconds later.
{code}
Feb 6, 2012 2:48:00 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel Chunked connection to http://99.99.99.94:8082/jenkins/cli
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
{code}
{code}
10.25.50.94 - - [06/Feb/2012:14:48:00 -0500] "POST /jenkins/cli HTTP/1.1" 200 13935
10.25.50.94 - - [06/Feb/2012:14:48:00 -0500] "POST /jenkins/cli HTTP/1.1" 200 -
{code}

The access log has a couple of extra cli commands in the meantime. It isn't clear whether these are for the same job or for something else. Assuming they are for something else that means that the 2nd command timed out roughly 10 seconds after the previous cli command.

{code}
10.25.50.94 - - [06/Feb/2012:14:47:39 -0500] "POST /jenkins/cli HTTP/1.1" 200 10156
10.25.50.94 - - [06/Feb/2012:14:47:39 -0500] "POST /jenkins/cli HTTP/1.1" 200 -

10.25.50.94 - - [06/Feb/2012:14:47:39 -0500] "POST /jenkins/cli HTTP/1.1" 200 10156
10.25.50.94 - - [06/Feb/2012:14:47:39 -0500] "POST /jenkins/cli HTTP/1.1" 200 -
{code}
I wonder if the http data upload inactivity timeout is 10 seconds in tomcat7.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159346#comment-159346 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

Problem found. Unfortunately it is deep in the emulator (qemu) startup code. The following code registers the emulator instance with the adb server.

{code}
qemu-setup.c:
sprintf(tmp,"0012host:emulator:%d",base_port+1);
socket_send(s, tmp, 18+4);
{code}

This registers the emulator with the adb server but it makes two assumptions:
# that the user port is one less than the adb port for the emulator. Comments in the code also suggest that the user port was once assumed to be an even number. The server takes the adb port number supplied and subtracts one to get the user port. It then tries to connect to the user port to confirm that it really is an emulator.
# only 4 characters of the adb port number is sent to the adb server (the +4 in socket_send). Thus if the port number is greater than 9999 or less than 1000 then the wrong port number is sent.

I haven't searched the Android bug database for this yet but will search/report there later today.

However that does not make things work for a-e-p because we will need to support older emulators.

I tested a fix that uses the Jenkins port allocator to allocate 7000 for user port, 7001 for emulator adb port and then 8000 for the adb server. With that maven happily worked using {{-Dandroid.device=emulator}}.

{code}
[INFO] Found 1 devices connected with the Android Debug Bridge
[INFO] android.device parameter set to emulator
[INFO] checking device=emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk, port=8000, name=hudson_en-GB_160_WVGA_android-7, isEmulator=true
[INFO] Emulator emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk found.
{code}

Obviously to provide a proper fix we need to do some magic with the port allocator plugin. Assuming that it is acceptable that we require a newer port allocator plugin for a-e-p then I'll look to extend that to help us. Failing that we can likely make the existing allocator work with a bit of messing around.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159346#comment-159346 ]

Richard Mortimer edited comment on JENKINS-12037 at 2/21/12 3:27 PM:
---------------------------------------------------------------------

Oops comment added to wrong issue!

was (Author: oldelvet):
Problem found. Unfortunately it is deep in the emulator (qemu) startup code. The following code registers the emulator instance with the adb server.

{code}
qemu-setup.c:
sprintf(tmp,"0012host:emulator:%d",base_port+1);
socket_send(s, tmp, 18+4);
{code}

This registers the emulator with the adb server but it makes two assumptions:
# that the user port is one less than the adb port for the emulator. Comments in the code also suggest that the user port was once assumed to be an even number. The server takes the adb port number supplied and subtracts one to get the user port. It then tries to connect to the user port to confirm that it really is an emulator.
# only 4 characters of the adb port number is sent to the adb server (the +4 in socket_send). Thus if the port number is greater than 9999 or less than 1000 then the wrong port number is sent.

I haven't searched the Android bug database for this yet but will search/report there later today.

However that does not make things work for a-e-p because we will need to support older emulators.

I tested a fix that uses the Jenkins port allocator to allocate 7000 for user port, 7001 for emulator adb port and then 8000 for the adb server. With that maven happily worked using {{-Dandroid.device=emulator}}.

{code}
[INFO] Found 1 devices connected with the Android Debug Bridge
[INFO] android.device parameter set to emulator
[INFO] checking device=emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk, port=8000, name=hudson_en-GB_160_WVGA_android-7, isEmulator=true
[INFO] Emulator emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk found.
{code}

Obviously to provide a proper fix we need to do some magic with the port allocator plugin. Assuming that it is acceptable that we require a newer port allocator plugin for a-e-p then I'll look to extend that to help us. Failing that we can likely make the existing allocator work with a bit of messing around.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159635#comment-159635 ]

mark streit commented on JENKINS-12037:
---------------------------------------

with regard to your question about the cli commands showing in the logs -

"The access log has a couple of extra cli commands in the meantime. It isn't clear whether these are for the same job or for something else. Assuming they are for something else that means that the 2nd command timed out roughly 10 seconds after the previous cli command."

This test was only done with no other jobs being run. Only the command shell being run that is running the jobs with the -s parameter. It "appears" to us that the -s parameter gets ignored and other jobs start running before we expect them to.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159639#comment-159639 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

If I build a test version for you with a 5 second heartbeat timeout are you able to test it? If so is there any particular Jenkins version that you would be testing with so that I can build a matching cli jar file.

Regarding the -s parameter being ignored I vaguely remember that there is a bug reported in JIRA where it does not wait if there is already a queued instance of the same job. From what you describe this may not be applicable to your circumstances but it could.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159874#comment-159874 ]

brianharris commented on JENKINS-12037:
---------------------------------------

Stack trace looks much like the popular JENKINS-6817.

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159883#comment-159883 ]

mcs130 commented on JENKINS-12037:
----------------------------------

Richard we obtained your 1.450 version of the JAR with heartbeat time interval change. We installed the 1.450 WAR into Tomcat 7.0.22 and run everything from jdk1.6.0_30 now - both the Tomcat startup and the JDK from which we run the CLI commands. We even set the connectionTimeout in the Tomcat server.xml from 20 seconds to as high as 60 seconds. Nothing is working - we still get the I/O chunked connection/Unexpected term of channel error. For now we have had to revert back to 1.441 running on Winstone only...and the original 1.441 CLI jar that came with it. When using this combination - it works but is not ideal. We can't get above 1.441 because even using Winstone with a later version fails in the same manner. And to your point, the -s parameter appears to get ignored when used. I will get the similar logs from the last experiment and send them as before.

Thanks

Mark

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159954#comment-159954 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

Well I think that pretty much rules out the ping interval directly. Obviously there may be something that is blocking the channel/pings. Alternately there may be something external that is breaking the connection. The logs will definitely be useful to confirm that it is still the upstream (CLI to Jenkins) connection that is being closed.

I'm not sure there is much more I can do without a reproducible test case that I can run here or unless someone who can reproduce can get a network packet dump (that they are willing to share) that will hopefully allow us to piece together what is going on at the network level.

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159955#comment-159955 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

@brianharris agree that the stacktrace looks similar. The one difference that I'm aware of is that the connections in JENKINS-12037 are done over the cli channel using two HTTP connections rather than to the JNLP port. In this case the HTTP container can break connections if it thinks that there is a "bad" connection and we could reproduce this particular failure and trace it back to that.

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160182#comment-160182 ]

Miguel Almeida commented on JENKINS-12037:
------------------------------------------

Hi,

I've just posted on the mailing list (http://groups.google.com/group/jenkinsci-users/browse_frm/thread/7f96cdf57ffed6cf# ) - perhaps this is the same issue.

I have a multi-module project where I'm running the Maven goal "clean
surefire-report:report".
While the tests pass in our local machines, we're randomly having failures
in one of the modules in Jenkins. The console shows:
{noformat}
mavenExecutionResult exceptions not empty
message : Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12:test
(default-test) on project cm-web: Error occurred in starting fork,
check output in log
cause : Error occurred in starting fork, check output in log
Stack trace :
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test
(default-test) on project cm-web: Error occurred in starting fork,
check output in log
{noformat}
We have Jenkins running in Tomcat and by running the build and inspecting
tomcat's log I see:

{noformat}
SEVERE: I/O error in channel Channel to Maven [java, -cp,
/usr/share/tomcat6/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.2.jar:/usr/share/tomcat6/.jenkins/tools/Jenkins_maven/boot/plexus-classworlds-2.4.jar,
org.jvnet.hudson.maven3.agent.Maven3Main,
/usr/share/tomcat6/.jenkins/tools/Jenkins_maven,
/usr/share/tomcat6/webapps/jenkins/WEB-INF/lib/remoting-2.12.jar,
/usr/share/tomcat6/.jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.2.jar,
40284]
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
at
java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)
{noformat}

Restarting tomcat makes the problem go away (until it "randomly" appears again after a few days)

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160200#comment-160200 ]

George Panayotov commented on JENKINS-12037:
--------------------------------------------

I'm running version 1.454 on OSX Lion via the .pkg install. I believe that means I have it running via WinStone. I have a similar issue to this (the same I think). I call build -s JOB_NAME via cli. The channel times out after 1 minute exactly.

Server log:

Mar 13, 2012 6:53:59 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel HTTP full-duplex channel fe769a28-6f68-4595-b207-be5ac016c678
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.net.SocketInputStream.read(SocketInputStream.java:182)
at winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)
at hudson.util.ChunkedInputStream.getChunkSizeFromInputStream(ChunkedInputStream.java:223)
at hudson.util.ChunkedInputStream.nextChunk(ChunkedInputStream.java:197)
at hudson.util.ChunkedInputStream.read(ChunkedInputStream.java:120)
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2248)
at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2541)
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2551)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)

Client Log:

Mar 13, 2012 6:53:59 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel Chunked connection to http://10.1.11.19:8080/cli
java.io.IOException: Premature EOF
at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538)
at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582)
at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2672)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2667)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2656)
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2248)
at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2541)
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2551)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)
Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
at hudson.remoting.Request.call(Request.java:149)
at hudson.remoting.Channel.call(Channel.java:681)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
at $Proxy2.main(Unknown Source)
at hudson.cli.CLI.execute(CLI.java:271)
at hudson.cli.CLI._main(CLI.java:417)
at hudson.cli.CLI.main(CLI.java:322)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
at hudson.remoting.Request.abort(Request.java:273)
at hudson.remoting.Channel.terminate(Channel.java:732)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1157)
Caused by: java.io.IOException: Premature EOF
at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538)
at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582)
at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2672)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2667)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2656)
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2248)
at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2541)
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2551)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)

I also connected WireShark to see if I could get any useful info from there and I noticed one thing among the mess of serialized data:

"Execution of hudson.remoting.PingThread$Ping@2be548d is prohibited because the channel is restricted"

I'm pretty sure my issue is that the server times out at 60 seconds and it seems like the line above might mean that the reason is that the timeout is set to 60 seconds and the PingThread is not running. Hope, this helps. Also, has anyone found a workaround for this yet?

-George

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160224#comment-160224 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

@Miguel Almeida thanks for the information. At the moment that seems sufficiently different to the situation/symptoms reported in this JIRA that I think it would be best if you could report that as a separate JIRA to stop confusion over symptoms. It may turn out to be related once the root cause is known as @brianharris points out.

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160225#comment-160225 ]

Richard Mortimer commented on JENKINS-12037:
--------------------------------------------

@George Panayotov thanks for the data. This does indeed look like similar symptoms to JENKINS-12037.

Thank you for taking a look using WireShark that might be a big clue as to what is going on. Looking at the source in [FullDuplexHttpChannel.java|https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/FullDuplexHttpChannel.java] that sets isRestricted based on user authentication so this could give a clue as to why it might occur for some and not others.

That said if you still have the Wireshark output around can you take a look to see if you can answer the following:
* Does the "is prohibited because the channel is restricted" message get repeated or does it just occur once at the start of the CLI session?
* Do you see any other activity on either of the two http streams (upstream and downstream) during the 60 seconds after the initial activity?

Richard

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

George Panayotov updated JENKINS-12037:
---------------------------------------

Attachment: jenkins-timeout

This is the Wireshark capture of the failing transaction. Hope this helps.

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: jenkins-timeout, Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-12037) CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-12037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160500#comment-160500 ]

George Panayotov commented on JENKINS-12037:
--------------------------------------------

I think the "Channel is restricted" only happened once. I don't think there was any other activity during the 60 seconds. You should check jenkins-timeout to see if you can find anything.

Thanks,
George

> CLI - I/O error in channel Chunked connection/Unexpected termination of the channel - still occurring in Jenkins 1.449
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: JENKINS-12037
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12037
> Project: Jenkins
> Issue Type: Bug
> Components: cli
> Environment: * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
> * Tomcat 7.0.22
> * JDK 1.6.0_14
> * Only ONE Master configuration - no slaves are configured
> * 3 Executors - (one less than the max number of CPUs)
> Reporter: mark streit
> Priority: Critical
> Attachments: jenkins-timeout, Tomcat7_Jenkins1449_logs.zip
>
>
> We reported an issue some time back that was also listed as fixed in Jenkins 1.441:
> Log:
> [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar
> Perhaps another bug should NOT be submitted so I have added the following comments below the line to the original defect 11130 comments section in case it can be reviewed/re-opened.
> We did NOT try to make any adjustments to the Tomcat configuration:
> Tomcat Connector connectionUploadTimeout
> but we are also now seeing the same problem with Winstone when at this same 1.441 level. We did revert to the 1.438 version of the CLI (leaving the WAR at 1.441 running in Winstone) and that is serving asthe current workaround.
> ================================================================================================
> We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).
> Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.
> Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Created-By: Apache Maven
> Built-By: kohsuke
> Build-Jdk: 1.6.0_26
> Main-Class: hudson.cli.CLI
> Jenkins-CLI-Version: 1.441
> Under Tomcat 7, we get this stacktrace:
> Started by command line
> [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Unexpected termination of the channel
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
> Caused by: java.io.EOFException
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE
> Under Winstone, we get this stacktrace - it's somewhat different:
> Started by command line
> [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
> + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
> Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
> java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.call(Request.java:149)
> at hudson.remoting.Channel.call(Channel.java:681)
> at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
> at $Proxy2.main(Unknown Source)
> at hudson.cli.CLI.execute(CLI.java:200)
> at hudson.cli.CLI._main(CLI.java:330)
> at hudson.cli.CLI.main(CLI.java:245)
> Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
> at hudson.remoting.Request.abort(Request.java:273)
> at hudson.remoting.Channel.terminate(Channel.java:732)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
> Caused by: java.io.IOException: Premature EOF
> at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (<a href="http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29">http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
> at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (<a href="http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29">http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
> at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (<a href="http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29">http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
> at java.io.FilterInputStream.read(FilterInputStream.java:116)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
> at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (<a href="http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29">http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
> at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
> at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
> at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
> at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
> Build step 'Execute shell' marked build as failure
> Notifying upstream projects of job completion
> Finished: FAILURE