Jenkins › Jenkins issues

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

Classic

List

Threaded

18 messages Options

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

Marcel Beister created JENKINS-13844:
----------------------------------------

Summary: "Emulate clean checkout by..." does not work with 1.7 subversion repositories
Key: JENKINS-13844
URL: https://issues.jenkins-ci.org/browse/JENKINS-13844
Project: Jenkins
Issue Type: Bug
Components: subversion
Affects Versions: current
Environment: Windows XP x64, Core i7
Reporter: Marcel Beister
Priority: Critical

The update strategy "Emulate clean checkout by..." does not delete files which are not part of the repository although the log reports that a cleanup is performed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

[ https://issues.jenkins-ci.org/browse/JENKINS-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=163017#comment-163017 ]

sogabe commented on JENKINS-13844:
----------------------------------

Subversion Plugin *DOES NOT* support 1.7.

> "Emulate clean checkout by..." does not work with 1.7 subversion repositories
> -----------------------------------------------------------------------------
>
> Key: JENKINS-13844
> URL: https://issues.jenkins-ci.org/browse/JENKINS-13844
> Project: Jenkins
> Issue Type: Bug
> Components: subversion
> Affects Versions: current
> Environment: Windows XP x64, Core i7
> Reporter: Marcel Beister
> Priority: Critical
> Labels: subversion
>
> The update strategy "Emulate clean checkout by..." does not delete files which are not part of the repository although the log reports that a cleanup is performed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

[ https://issues.jenkins-ci.org/browse/JENKINS-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=163043#comment-163043 ]

Marcel Beister commented on JENKINS-13844:
------------------------------------------

To clarify this: The error happens when I chose "1.7" as format for the working copy, the server runs subversion "1.6".

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Same for me using SVN server 1.5, Jenkins 1.472 + SVN 1.42 plugin, SVN WC 1.7
Issue occurs on a WinXP 32bit slave

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

same here, environmen same as in comment above me, only I have jenkins 1.475 and it's a master building node

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

This is caused by changes in SVN 1.7 relating to ignored files - they are no longer reported by default in 'svn status' (the SVNKit API used by Jenkins mirrors this behaviour).

In the workspace for a Jenkins project (after the job has run), no changes are shown:

[root@devemr01 UTIL_TRUNK]# svn status

However, the below command shows files/directories that are marked as ignored:

[root@devemr01 UTIL_TRUNK]# svn status --no-ignore
I       .gradle
I       build

...and creating a new file that isn't ignored shows up in 'svn status':

[root@devemr01 UTIL_TRUNK]# touch foo
[root@devemr01 UTIL_TRUNK]# svn status
?       foo

UpdateWithCleanUpdater.java needs to explicitly request that SVNKit include 'ignored' files such that they can be deleted.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Code from UpdateWithCleanUpdater.java:

clientManager.getStatusClient().doStatus(local, null, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler() {
                public void handleStatus(SVNStatus status) throws SVNException {
                    SVNStatusType s = status.getContentsStatus();
                    if (s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED) {
                        listener.getLogger().println("Deleting "+status.getFile());
                        try {
                            File f = status.getFile();
                            if (f.isDirectory())
                                hudson.Util.deleteRecursive(f);
                            else
                                f.delete();
                        } catch (IOException e) {
                            throw new SVNException(SVNErrorMessage.create(SVNErrorCode.UNKNOWN, e));
                        }
                    }
                }
            }, null);

...the fifth parameter is 'includeIgnored' which is strangely set to false (and always has been), although the subsequent code expects ignored elements. Guessing that the inclusion of 'ignored' files was a bug with previous SVN (and SVNKit) versions that Jenkins inadvertently relied on.

SVNKit API: http://svnkit.com/javadoc/org/tmatesoft/svn/core/wc/SVNStatusClient.html#doStatus(java.io.File, org.tmatesoft.svn.core.wc.SVNRevision, org.tmatesoft.svn.core.SVNDepth, boolean, boolean, boolean, boolean, org.tmatesoft.svn.core.wc.ISVNStatusHandler, java.util.Collection)

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Does this explain not deleting unversioned files as well? Because you're mentioning only ignored files...

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

No. There is more to this issue than just the ignored files.

Here's a test case from a simple Jenkins / SVN (Jenkins 1.483, SVN 1.7):

After wiping out the workspace and running an initial build:

[root@devemr01 UTIL_TRUNK]# ll
total 24
drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build
-rw-rw-r--  1 jenkins jenkins  376 Sep 30 00:06 build.gradle
drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 lib
drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 releng
-rw-rw-r--  1 jenkins jenkins   37 Sep 30 00:06 settings.gradle
drwxrwxr-x  4 jenkins jenkins 4096 Sep 30 00:06 src

...adding an unversioned file:

[root@devemr01 UTIL_TRUNK]# touch foo
[root@devemr01 UTIL_TRUNK]# svn status
?       foo

Running the job (note that the unversioned file wasn't detected or removed):

00:07:59 Cleaning up /home/jenkins/workspace/UTIL_TRUNK/.
00:07:59 Updating svn://svn.ma.net/util/trunk to revision '2012-09-30T00:07:58.927 -0700'
00:07:59 At revision 38394

...and the unversioned file, and the ignored dir 'build', are still present:

[root@devemr01 UTIL_TRUNK]# ll
total 24
drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build
-rw-rw-r--  1 jenkins jenkins  376 Sep 30 00:06 build.gradle
-rw-r--r--  1 root    root       0 Sep 30 00:07 foo
drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 lib
drwxrwxr-x  2 jenkins jenkins 4096 Sep 30 00:06 releng
-rw-rw-r--  1 jenkins jenkins   37 Sep 30 00:06 settings.gradle
drwxrwxr-x  4 jenkins jenkins 4096 Sep 30 00:06 src
[root@devemr01 UTIL_TRUNK]#

It appears that the SVNKit 'status' API call is working differently (and possibly incorrectly) for SVN 1.7 repositories.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Found the issue; the status objects returned by SVNKit have different values when used against a SVN 1.7 working copy.

Using the original snippet of code, status objects are returned for the correct files, but their status is STATUS_NONE, and hence the subsequent logic for removing files of specific statuses is not triggered:

/home/jenkins/workspace/UTIL_TRUNK/.gradle    none 
/home/jenkins/workspace/UTIL_TRUNK/foo        none 
/home/jenkins/workspace/UTIL_TRUNK/build      none

Modifying the code to combine the node & contents status yields the correct results:

/home/jenkins/workspace/UTIL_TRUNK/.gradle                   ignored
Deleting /home/jenkins/workspace/UTIL_TRUNK/.gradle
/home/jenkins/workspace/UTIL_TRUNK/foo                       unversioned
Deleting /home/jenkins/workspace/UTIL_TRUNK/foo
/home/jenkins/workspace/UTIL_TRUNK/build                     ignored

The relevant change is:

// for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.) 
     SVNStatusType s = status.getCombinedNodeAndContentsStatus(); 
     //SVNStatusType s = status.getContentsStatus();

The complete test harness is:

import java.io.File;

import java.util.*;
import org.tmatesoft.svn.core.SVNDepth;
import org.tmatesoft.svn.core.SVNException;
import org.tmatesoft.svn.core.wc.ISVNStatusHandler;
import org.tmatesoft.svn.core.wc.SVNClientManager;
import org.tmatesoft.svn.core.wc.SVNStatus;
import org.tmatesoft.svn.core.wc.SVNStatusType;
import org.tmatesoft.svn.core.wc.*;
import org.tmatesoft.svn.core.internal.wc.*;

public class Test
{
    public static void main( String[] args ) throws SVNException
    {
        SVNClientManager clientManager = SVNClientManager.newInstance();

        Collection<String> changeLists = Collections.emptyList();
        //clientManager.getStatusClient().doStatus( new File( args[ 0 ] ).getAbsoluteFile(), SVNRevision.WORKING, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler()
        clientManager.getStatusClient().doStatus( new File( args[ 0 ] ), null, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler()
        {
            @Override
            public void handleStatus( SVNStatus status ) throws SVNException
            {
                // for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.)
                SVNStatusType s = status.getCombinedNodeAndContentsStatus();
                //SVNStatusType s = status.getContentsStatus();

                System.out.printf( "%-60s %s\n", status.getFile(), s );
                if( s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED )
                {
                    System.out.println( "Deleting " + status.getFile() );
                }
            }
        }, null );
    }
}

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Code changed in jenkins
User: Nicolas De loof
Path:
src/main/java/hudson/scm/subversion/UpdateWithCleanUpdater.java
http://jenkins-ci.org/commit/subversion-plugin/82377581f16cccaed8c4a90169d7d4c11d3f9845
Log:
Merge pull request #22 from forderud/master

JENKINS-13844: "Emulate clean checkout by..." does not work with 1.7 subversion repositories

Compare: https://github.com/jenkinsci/subversion-plugin/compare/3f18badadf37...82377581f16c

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Unable to reproduce problem anymore after commit of pull request #22. Therefore resolving.
Change By:	Fredrik Orderud (29/Oct/12 7:57 AM)
Status:	Open Resolved
Resolution:	Fixed

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Still happens on 1.492 with svn 1.43.

System:
java.vm.version 22.1-b02
line.separator
mail.smtp.sendpartial true
mail.smtps.sendpartial true
os.arch amd64
os.name Windows Server 2008 R2 (Datacenter)

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

The fix has now been released with Subversion plugin 1.44.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

The fix has now been released with Subversion plugin 1.44.

Linards L: Could you please try to upgrade to SVN plugin 1.44, and try to reproduce afterwards?

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Ok. Please update fixVersion field

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

Agree, but for it appears like the version field(s) aren't properly configured. I receive an "1.44 is not a valid version" error message when attempting to update the field.

JIRA noreply@jenkins-ci.org

[JIRA] (JENKINS-13844) "Emulate clean checkout by..." does not work with 1.7 subversion repositories

In reply to this post by JIRA noreply@jenkins-ci.org

The version fields in JIRA only have the Jenkins core versions. This is a known limitation of the current JIRA setup.