On Thu, 18 Jun 2009 14:28:13 -0300, Sidnei da Silva wrote:

Hi Wolfgang,

On Thu, Jun 18, 2009 at 7:31 AM, Wolfgang Schnerring<ws@gocept.com> wrote:

...

* Sebastien Douche <sdouche@gmail.com> [2009-06-18 01:34]:

...

This is a first attempt to build an Mercurial mirror : http://hg.zope.mirrors.securactive.org/

How did you convert the repository?

I'm asking because I noticed that basically all SVN->DVCS conversion tools (hg convert, git-svn, bzr svn-import, svn2bzr, svn-fast-export.py, svn-all-fast-export.cpp) do not convert the history properly, more precisely, history that happened on branches is lost:

1. import /trunk/foo.txt 2. branch /trunk to /branches/mybranch 3. edit /mybranch/foo.txt 4. merge /mybranch to /trunk

If you now ask svn for the history of /trunk/foo.txt (say with 'svn log'), you see both steps 3 and 4. After conversion to a DVCS with one of the above mentioned tools, you only see step 4, while step 3 never happened in the DVCS repository. I think that's unacceptable, most importantly because all commit messages that happened on branches are lost that way.

Does somebody here know something about this phenomenon, by any chance? Am I missing something?

Here's some context about this from one of the Bazaar developers, John Arbash Meinel. Hopefully that will solve some of your questions?

""" In pretty much all dvcs merging a content exactly back to trunk does not generate a change message when doing "bzr log foo.txt".

I'm not really sure what he means by "edit" and then "merge" without a commit inbetween. So I'm assuming there is one.

Now, what really matters is whether or not *Subversion* recorded 4 correctly, such that it can actually see that it was a merge from 3.

My understanding is that before svn 1.5 that isn't possible. So you are left with trying to infer that sort of thing from the history. Which would be possible, but probably expensive.

I'm pretty sure SVN represents (4) as not a *merge* but as an indentical commit.

I don't have a great answer there. Though the fact that Wolfgang says svn shows both... I suppose because "svn log" shows everything across all branches? """

I'm somewhat confused here. According to my understanding: - all DVCS shows, if a merge is done, the changesets that origin from the merged branch. (this is the normal operation) - afaik svn does _not_ show this, what's more, it does not store any metadata about the merges or changesets involved. When doing the merge you really select a diff of the branch by specifying which changesets you want to include back in trunk. This is why it's so important with svn to note in the commit message, which revisions from which branch you merged. Otherwise you would not know at all what has been merged. So although, DVCS could represent the information about the merged changesets, this information will not be imported from SVN, simply, because the information is not represented in SVN. I'd like to add that I'm not using Hg, I am only using bazaar and svn, and I'm talking from what I experienced in practice with working on various svn repos and bzr. It's possible that newer svn does try to attack this problem by storing more metadata with the merges, which then would make sense to be considered at a DVCS import, but I believe that in the vast majority of svn repositories that you would consider importing, this information would not be there anyway, due to the fact that they are product of the older svn version. -- Balazs Ree

Hi there, * Sidnei da Silva <sidnei.da.silva@gmail.com> [2009-06-18 14:28]:

...

I'm asking because I noticed that basically all SVN->DVCS conversion tools (hg convert, git-svn, bzr svn-import, svn2bzr, svn-fast-export.py, svn-all-fast-export.cpp) do not convert the history properly, more precisely, history that happened on branches is lost:

Here's some context about this from one of the Bazaar developers, John Arbash Meinel. Hopefully that will solve some of your questions?

Thanks for relaying!

I'm not really sure what he means by "edit" and then "merge" without a commit inbetween. So I'm assuming there is one.

Sorry for being overly brief in my description; please find below a sample shell script to set up a Subversion repository with a project containing a trunk and a branch. To reproduce the problem I'm concerned about, do this: $ create-repos.sh repos $ svn log repos/project/trunk/feature.txt ------------------------------------------------------------------------ r9 | wosc | 2009-06-18 17:11:46 +0200 (Thu, 18 Jun 2009) | 1 line merged feature1 ------------------------------------------------------------------------ r8 | wosc | 2009-06-18 17:11:43 +0200 (Thu, 18 Jun 2009) | 1 line implemented feature1 ------------------------------------------------------------------------ As you can see, Subversion reports the history of the file that happenened on the branch (in r8). After converting the repository, however... $ bzr init-repo repos-bzr $ cd repos-bzr $ bzr svn-import file://$PWD/../repos $ bzr log repos/project/trunk/feature.txt ------------------------------------------------------------ revno: 3 svn revno: 9 (on /project/trunk) committer: wosc timestamp: Thu 2009-06-18 14:11:46 +0000 message: merged feature1 ... this history is lost. Not only does it not appear in the log output, also when looking at bzr visualize it seems like that branch never existed at all in bzr. (I'm new to bzr, am I missing something here?)

Now, what really matters is whether or not *Subversion* recorded 4 correctly, such that it can actually see that it was a merge from 3. My understanding is that before svn 1.5 that isn't possible.

My understanding is quite different. ;-) As far as I know, SVN "always" has recorded the history of a file across copies and merges, and so I'm quite sure this has nothing to do with 1.5-style merge tracking. I won't try to track down a reference on that right now, but I'm currently reading up on SVN's APIs so I expect to find the answer there, anyway.

I'm pretty sure SVN represents (4) as not a *merge* but as an indentical commit.

As I said, I'm still digging through the SVN APIs, so unfortunately I don't have a concrete answer, but I know positively that this history information is readily available, which you can see by passing --verbose to 'svn log': $ svn log -v -c 9 repos ------------------------------------------------------------------------ r9 | wosc | 2009-06-18 17:11:46 +0200 (Thu, 18 Jun 2009) | 1 line Changed paths: M /project/trunk A /project/trunk/feature.txt (from /project/branches/feature1/feature.txt:8) >>>>>>>>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ M /project/trunk/foo.txt merged feature1 ------------------------------------------------------------------------ Wolfgang ** #!/bin/sh if [ $# -lt 1 ]; then echo 1>&2 "Usage: $0 repos-path" exit 1 fi set -e mkdir $1 repos=$(cd "$1" && pwd) rmdir $1 working=$(tempfile) rm $working mkdir $working svnadmin create $repos svn mkdir file://$repos/project -m "creating project home" svn mkdir file://$repos/project/trunk -m "creating trunk" svn mkdir file://$repos/project/branches -m "creating branches" svn mkdir file://$repos/project/tags -m "creating tags" svn co file://$repos/project/trunk $working cd $working mkdir alpha echo "foo" > foo.txt echo "bar" > alpha/bar.txt svn add foo.txt alpha svn commit -m "initial version" svn cp file://$repos/project/trunk file://$repos/project/tags/0.1 -m "tagging 0.1" svn cp file://$repos/project/trunk file://$repos/project/branches/feature1 -m "creating branch 1" svn switch file://$repos/project/branches/feature1 echo "qux" > foo.txt echo "feature1" > feature.txt svn add feature.txt svn commit -m "implemented feature1" svn switch file://$repos/project/trunk svn merge file://$repos/project/branches/feature1 svn commit -m "merged feature1" svn rm file://$repos/project/branches/feature1 -m "remove branch after merge" svn cp file://$repos/project/trunk file://$repos/project/tags/0.2 -m "tagging 0.2" svn cp file://$repos/project/trunk file://$repos/project/branches/feature2 -m "creating branch 2" svn switch file://$repos/project/branches/feature2 echo "feature2" >> feature.txt svn commit -m "implemented feature 2" rm -rf $working

Hello, * Sidnei da Silva <sidnei.da.silva@gmail.com> [2009-06-19 14:26]:

On Fri, Jun 19, 2009 at 2:43 AM, Wolfgang Schnerring<ws@gocept.com> wrote:

...

$ svn log repos/project/trunk/feature.txt ------------------------------------------------------------------------ r9 | wosc | 2009-06-18 17:11:46 +0200 (Thu, 18 Jun 2009) | 1 line

merged feature1 ------------------------------------------------------------------------ r8 | wosc | 2009-06-18 17:11:43 +0200 (Thu, 18 Jun 2009) | 1 line

implemented feature1 ------------------------------------------------------------------------

As you can see, Subversion reports the history of the file that happenened on the branch (in r8).

Can you show us your 'svn --version'? I suspect what you're seeing is a result of svn 1.5 merge tracking.

I've dug a little deeper, and found that the example I presented is a special case, since 'feature.txt' has been *added* on the branch, and Subversion has indeed always treated the history of copied/added/removed files differently. You're right that in the general case, Subversion is only able to show the whole history (including branches) using the 1.5-mergeinfo support. One can invoke this via 'svn log --use-merge-history': on my example repository, compare the output for project/trunk/foo.txt with and without that switch; without it only shows the merge commit. Hmm. This means that even if conversion tools did something useful with the mergeinfo properties (which right now they don't), it would only solve my specific problem of preserving history for the last 6-12 months or so, since the repository I'm dealing with stems from 2003 or something, meaning Subversion 1.2ish, which did not even *store* the information necessary to recover the history. I guess I'll need to think about this some more (and then move to a different mailing list, as we're getting wayyyyy off-topic for zope-dev ;-) Thanks for your insights, Wolfgang