Status of github migration
Hi, I wanted to fix some logging in Products.SiteErrorLog, but I am confused where to go to. zopefoundation on github has a small number of repos but not Products.SiteErrorLog. http://svn.zope.org shows me a web view of our old cvs server. There is something wrong. Can I help with something? Best regards, Patrick
Le 10/01/2013 11:23, Patrick Gerken a écrit :
Hi,
I wanted to fix some logging in Products.SiteErrorLog, but I am confused where to go to.
here it is, part of Zope module : http://svn.zope.org/Zope/trunk/src/Products/SiteErrorLog/
On Jan 10, 2013, at 11:23 , Patrick Gerken <do3ccqrv@googlemail.com> wrote:
I wanted to fix some logging in Products.SiteErrorLog, but I am confused where to go to.
zopefoundation on github has a small number of repos but not Products.SiteErrorLog. http://svn.zope.org shows me a web view of our old cvs server.
There is something wrong. Can I help with something?
Going to http://svn.zope.org works fine for me. I see the expected SVN repositories. The GitHub migration happens on an as-needed basis. Package maintainers may request to have packages migrated or, like Jim and Tres are already doing, migrate packages themselves. There is no full migration of all svn.zope.org content. For those packages that are fully migrated you will have obvious markers on the package in svn.zope.org, such as the package being read-only. jens
Following your link it works for me too. Turned out I use a firefox plugin that if I enter a url without specifying a protocol, it tries https first. https://svn.zope.org redirects to the cvs. Regarding repos, I once did a full git mirror of the full svn.zope.org. I try to see if I can recreate single repos from it by moving stuff around and modifying the history. git svn clone is dog slow with svn.zope.org. On Thu, Jan 10, 2013 at 11:48 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
On Jan 10, 2013, at 11:23 , Patrick Gerken <do3ccqrv@googlemail.com> wrote:
I wanted to fix some logging in Products.SiteErrorLog, but I am confused where to go to.
zopefoundation on github has a small number of repos but not Products.SiteErrorLog. http://svn.zope.org shows me a web view of our old cvs server.
There is something wrong. Can I help with something?
Going to http://svn.zope.org works fine for me. I see the expected SVN repositories.
The GitHub migration happens on an as-needed basis. Package maintainers may request to have packages migrated or, like Jim and Tres are already doing, migrate packages themselves. There is no full migration of all svn.zope.org content.
For those packages that are fully migrated you will have obvious markers on the package in svn.zope.org, such as the package being read-only.
jens
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
On Thu, Jan 10, 2013 at 11:48 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
The GitHub migration happens on an as-needed basis.
Don't see the point, why not migrate all active packages (aka the ZTK + ZopeApp)? -- Sebastien Douche <sdouche@gmail.com> Twitter: @sdouche / G+: +sdouche
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/10/2013 07:02 AM, Sebastien Douche wrote:
On Thu, Jan 10, 2013 at 11:48 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
The GitHub migration happens on an as-needed basis.
Don't see the point, why not migrate all active packages (aka the ZTK + ZopeApp)?
Because it is a helluva lot of work, which can't be trivially scripted (things can go wrong: each project needs a person who knows it well to review the migrated repo). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlDu86MACgkQ+gerLs4ltQ5jMQCfShe7ev+ti+PAS66pXOPVLRZ6 Qy4AoNu3SxntlWgSfMfc7UbkHJ5fQRUo =maE3 -----END PGP SIGNATURE-----
Tres Seaver wrote:
Because it is a helluva lot of work, which can't be trivially scripted (things can go wrong: each project needs a person who knows it well to review the migrated repo).
When Plone did this the people involved wrote some scripts https://github.com/plone/svn-migrate I'm sure they'd be willing to help move all of Zope across, too. Should I have a word? Matt
On Jan 10, 2013, at 21:02, Matthew Wilkes <matthew@matthewwilkes.co.uk> wrote:
When Plone did this the people involved wrote some scripts
That stuff works only partially. It uses a GitHub API that has been removed months ago. I migrated the Zope package based on that software and apparently there are issues with missing branches, too. jens
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/10/2013 03:02 PM, Matthew Wilkes wrote:
Tres Seaver wrote:
Because it is a helluva lot of work, which can't be trivially scripted (things can go wrong: each project needs a person who knows it well to review the migrated repo).
When Plone did this the people involved wrote some scripts
What is needed is not scripts, but eyeballs: we need people who know the various packages and *care* about getting them migrated to github to step up. Softwward which doesn't have a champion willing to do the work should stay behind on SVN. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlDvQiMACgkQ+gerLs4ltQ5dAwCgwtWbn9IRnpjsD3AxYlgkLmZs PT0AoKd/cY6c0cJ8z4DzMZ+w0qcXtTLv =S8Qf -----END PGP SIGNATURE-----
Tres Seaver wrote:
What is needed is not scripts, but eyeballs: we need people who know the various packages and*care* about getting them migrated to github to step up. Softwward which doesn't have a champion willing to do the work should stay behind on SVN.
The community as a whole cares about having them all migrated to github. I'm sure this will happen the next time there's a sprint, just like lots of them got migrated (and subsequently deleted) at the zope4 sprint in San Francisco a few years back. We need man-hours, sure, but not champions. Being blocked on working on the code because you're the first one to care about a package and subsequently have to learn how to do the migration is a crazy way of doing things. Matthew
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/10/2013 06:10 PM, Matthew Wilkes wrote:
Tres Seaver wrote:
What is needed is not scripts, but eyeballs: we need people who know the various packages and*care* about getting them migrated to github to step up. Softwward which doesn't have a champion willing to do the work should stay behind on SVN.
The community as a whole cares about having them all migrated to github. I'm sure this will happen the next time there's a sprint, just like lots of them got migrated (and subsequently deleted) at the zope4 sprint in San Francisco a few years back.
The communite as-a-whole demonstrably does *not* care about many of the projects on svn.zope.org. E.g.: https://mail.zope.org/pipermail/zope-tests/2013-January/070977.html
We need man-hours, sure, but not champions. Being blocked on working on the code because you're the first one to care about a package and subsequently have to learn how to do the migration is a crazy way of doing things.
The foundation agreed to support moving projects to github, but that isn't a blank check. For instance, if there is substantial interest in having the projects pulled in by the current Plone buildout moved, make a list of them, and recruit the folks to step up and help with the migration for them. The effort requires includes doing the conversion, checking the results *by hand*, landing the repository, and fixing anything that breaks once you do (including stuff that breaks in projects you otherwise don't care about). Any project that can't find somebody willing to do that work (that is what I meant by a "champion") is better off staying on SVN: we don't do ourselves favors by carrying all the unmaintained baggage of fifteen years worth of development forward, just for "purity" / completeness / whatever. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlDwMPQACgkQ+gerLs4ltQ6HowCgse8NF8ELeMXSLB4USzBJD1mE mRAAnRU1bygjDMqeb3rn/674V/FfuvZY =Erwd -----END PGP SIGNATURE-----
On Fri, Jan 11, 2013 at 10:34 AM, Tres Seaver <tseaver@palladion.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01/10/2013 06:10 PM, Matthew Wilkes wrote:
Tres Seaver wrote:
What is needed is not scripts, but eyeballs: we need people who know the various packages and*care* about getting them migrated to github to step up. Softwward which doesn't have a champion willing to do the work should stay behind on SVN.
The community as a whole cares about having them all migrated to github. I'm sure this will happen the next time there's a sprint, just like lots of them got migrated (and subsequently deleted) at the zope4 sprint in San Francisco a few years back.
The communite as-a-whole demonstrably does *not* care about many of the projects on svn.zope.org. E.g.:
https://mail.zope.org/pipermail/zope-tests/2013-January/070977.html
We need man-hours, sure, but not champions. Being blocked on working on the code because you're the first one to care about a package and subsequently have to learn how to do the migration is a crazy way of doing things.
The foundation agreed to support moving projects to github, but that isn't a blank check. For instance, if there is substantial interest in having the projects pulled in by the current Plone buildout moved, make a list of them, and recruit the folks to step up and help with the migration for them. The effort requires includes doing the conversion, checking the results *by hand*, landing the repository, and fixing anything that breaks once you do (including stuff that breaks in projects you otherwise don't care about).
Any project that can't find somebody willing to do that work (that is what I meant by a "champion") is better off staying on SVN: we don't do ourselves favors by carrying all the unmaintained baggage of fifteen years worth of development forward, just for "purity" / completeness / whatever.
+1 BTW (speaking of cruft), as someone who used/abused svn.zope.org as a generic open-source hosting service (when I should have used something like code.google.com, or bitbucket, or whatever), I wonder if there should be a process for petitioning to remove projects from the ZF repositories. (Maybe this only applies to me :) Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
Hi everybody, I made what necessary for z3c.soap to comply with the zope project policy (used zope.repositorypolicy) and incindentally to work with Zope 2.13.19, what is the process now so it is migrated to Github? Thank you and have a nice day, Gauthier Bastien Support IMIO - CommunesPlone rue de la Vieille Sambre 34 5190 Mornimont Tél: +32(0)71 780979 Disclaimer Les informations contenues dans ce courrier électronique (annexes incluses) sont confidentielles et réservées à l'usage exclusif des destinataires repris ci-dessus. Si vous n'êtes pas le destinataire, soyez informé par la présente que vous ne pouvez ni divulguer, ni reproduire, ni faire usage de ces informations pour vous-même ou toute tierce personne. Si vous avez reçu ce courrier électronique par erreur, vous êtes prié d'en avertir immédiatement l'expéditeur et d'effacer le message e-mail de votre ordinateur. De informatie in deze e-mail, bijlagen inbegrepen, is vertrouwelijk en is als dus danig voorbehouden voor exclusief gebruik door de hierboven vermelde bestemmeling(en). Indien u niet de bestemmeling bent, willen wij u erop wijzen dat u deze informatie niet mag aanwenden voor eigen gebruik noch verspreiden aan derden. Indien u deze e-mail per ongeluk hebt ontvangen, gelieve de afzender onmiddellijk te verwittigen en deze e-mail van uw computer te verwijderen. The information contained in this e-mail and the annexed documents is confidential and exclusively available to the here above mentioned addressee(s).Should you not be the addressee, please be informed that you may neither disclose nor reproduce this e-mail, nor may the information contained in this e-mail and its eventually annexed documents be used by yourself or by a third party. If you erroneously received this e-mail, could you kindly and immediately inform the addresser and delete the message on your computer. Le 11/01/13 16:42, Jim Fulton a écrit :
On Fri, Jan 11, 2013 at 10:34 AM, Tres Seaver <tseaver@palladion.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01/10/2013 06:10 PM, Matthew Wilkes wrote:
Tres Seaver wrote:
What is needed is not scripts, but eyeballs: we need people who know the various packages and*care* about getting them migrated to github to step up. Softwward which doesn't have a champion willing to do the work should stay behind on SVN. The community as a whole cares about having them all migrated to github. I'm sure this will happen the next time there's a sprint, just like lots of them got migrated (and subsequently deleted) at the zope4 sprint in San Francisco a few years back. The communite as-a-whole demonstrably does *not* care about many of the projects on svn.zope.org. E.g.:
https://mail.zope.org/pipermail/zope-tests/2013-January/070977.html
We need man-hours, sure, but not champions. Being blocked on working on the code because you're the first one to care about a package and subsequently have to learn how to do the migration is a crazy way of doing things. The foundation agreed to support moving projects to github, but that isn't a blank check. For instance, if there is substantial interest in having the projects pulled in by the current Plone buildout moved, make a list of them, and recruit the folks to step up and help with the migration for them. The effort requires includes doing the conversion, checking the results *by hand*, landing the repository, and fixing anything that breaks once you do (including stuff that breaks in projects you otherwise don't care about).
Any project that can't find somebody willing to do that work (that is what I meant by a "champion") is better off staying on SVN: we don't do ourselves favors by carrying all the unmaintained baggage of fifteen years worth of development forward, just for "purity" / completeness / whatever. +1
BTW (speaking of cruft), as someone who used/abused svn.zope.org as a generic open-source hosting service (when I should have used something like code.google.com, or bitbucket, or whatever), I wonder if there should be a process for petitioning to remove projects from the ZF repositories. (Maybe this only applies to me :)
Jim
On Thu, Jan 10, 2013 at 5:48 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
On Jan 10, 2013, at 11:23 , Patrick Gerken <do3ccqrv@googlemail.com> wrote:
I wanted to fix some logging in Products.SiteErrorLog, but I am confused where to go to.
zopefoundation on github has a small number of repos but not Products.SiteErrorLog. http://svn.zope.org shows me a web view of our old cvs server.
There is something wrong. Can I help with something?
Going to http://svn.zope.org works fine for me. I see the expected SVN repositories.
The GitHub migration happens on an as-needed basis. Package maintainers may request to have packages migrated or, like Jim and Tres are already doing, migrate packages themselves. There is no full migration of all svn.zope.org content.
For those packages that are fully migrated you will have obvious markers on the package in svn.zope.org, such as the package being read-only.
I don't think being read only is enough. Are people supposed to attempt commits to find out if a project is read-only? We should at least leave something like a MOVED_TO_GITHUB file in the project, in addition to making it read only. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On Jan 10, 2013, at 14:37 , Jim Fulton <jim@zope.com> wrote:
On Thu, Jan 10, 2013 at 5:48 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
For those packages that are fully migrated you will have obvious markers on the package in svn.zope.org, such as the package being read-only.
I don't think being read only is enough. Are people supposed to attempt commits to find out if a project is read-only?
We should at least leave something like a MOVED_TO_GITHUB file in the project, in addition to making it read only.
That's what I meant by "obvious marker, such as...". I did not imply making a package read-only is the only marker. jens
On Thu, Jan 10, 2013 at 8:39 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
On Jan 10, 2013, at 14:37 , Jim Fulton <jim@zope.com> wrote:
On Thu, Jan 10, 2013 at 5:48 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
For those packages that are fully migrated you will have obvious markers on the package in svn.zope.org, such as the package being read-only.
I don't think being read only is enough. Are people supposed to attempt commits to find out if a project is read-only?
We should at least leave something like a MOVED_TO_GITHUB file in the project, in addition to making it read only.
That's what I meant by "obvious marker, such as...". I did not imply making a package read-only is the only marker.
But that's all that has been done for some projects. (Sorry, I don't mean to be critical and I didn't raise the issue when it happened.) OTOH, it's been argued (I disagree :) that I did too much for the ZODB projects. I think it would be helpful if we agreed on what should done, so we have a standard play book. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On Jan 10, 2013, at 14:45 , Jim Fulton <jim@zope.com> wrote:
On Thu, Jan 10, 2013 at 8:39 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
On Jan 10, 2013, at 14:37 , Jim Fulton <jim@zope.com> wrote:
On Thu, Jan 10, 2013 at 5:48 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
For those packages that are fully migrated you will have obvious markers on the package in svn.zope.org, such as the package being read-only.
I don't think being read only is enough. Are people supposed to attempt commits to find out if a project is read-only?
We should at least leave something like a MOVED_TO_GITHUB file in the project, in addition to making it read only.
That's what I meant by "obvious marker, such as...". I did not imply making a package read-only is the only marker.
But that's all that has been done for some projects. (Sorry, I don't mean to be critical and I didn't raise the issue when it happened.) OTOH, it's been argued (I disagree :) that I did too much for the ZODB projects. I think it would be helpful if we agreed on what should done, so we have a standard play book.
Please get in touch with Tres if you think his migrations should be improved. All other repositories (Zope, Products.*) were test migrations where I asked for feedback and never got any. They are throw-away and not final. The only finished migrations are yours and Tres'. jens
Hi Jens, On Thu, Jan 10, 2013 at 11:47 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
[...] All other repositories (Zope, Products.*) were test migrations where I asked for feedback and never got any. They are throw-away and not final. The only finished migrations are yours and Tres'.
I've been overly busy with these last couple of monks with job change/country relocation/new year, so I couldn't review those migrations before. I took a quick look at the Zope migration now and I think it's excellent. The only thing I'd add is that I'd also migrate branches 2.12 and 2.13 branches since they're all active, even if just for bug/security fixes. Having those branches means that back/forward-porting fixes is easier. Although we could always recreate the branches based on the tags, which were also nicely migrated. I think you mentioned this before, but I couldn't find it in a quick archive search, but which script did you use for those migrations? Thanks for all the work! And thanks also for the very nice mirror at: git.zope.org Cheers, Leo
jens
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
On Jan 10, 2013, at 15:40 , Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
I took a quick look at the Zope migration now and I think it's excellent. The only thing I'd add is that I'd also migrate branches 2.12 and 2.13 branches since they're all active, even if just for bug/security fixes.
I did not choose to include or exclude any branch. The test migration uses the package used to migrate most Plone packages from SVN to GitHub, which uses svn2git underneath. If there's whole branches missing the migration has obviously failed. If you want to help you could try a migration with the attached Python script that Jim wrote as a result of finding bugs in svn2git. I can provide you with a suitable authors mapping file. jens
On Thu, Jan 10, 2013 at 9:50 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
On Jan 10, 2013, at 15:40 , Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
I took a quick look at the Zope migration now and I think it's excellent. The only thing I'd add is that I'd also migrate branches 2.12 and 2.13 branches since they're all active, even if just for bug/security fixes.
I did not choose to include or exclude any branch. The test migration uses the package used to migrate most Plone packages from SVN to GitHub, which uses svn2git underneath. If there's whole branches missing the migration has obviously failed.
I had a bunch of problems with ZODB and the ruby svn2git. That's why I wrote my own based on seeing what the ruby version was trying to do. Note that most of the heavy lifting is actually done by the git svn plugin.
If you want to help you could try a migration with the attached Python script that Jim wrote as a result of finding bugs in svn2git. I can provide you with a suitable authors mapping file.
I've been waiting to make the script more publicly available in a saner fashion until I've worked with it some more. I should just make it a gist. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On Thu, Jan 10, 2013 at 3:59 PM, Jim Fulton <jim@zope.com> wrote:
On Thu, Jan 10, 2013 at 9:50 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
I did not choose to include or exclude any branch. The test migration uses the package used to migrate most Plone packages from SVN to GitHub, which uses svn2git underneath. If there's whole branches missing the migration has obviously failed.
I had a bunch of problems with ZODB and the ruby svn2git.
If you used a ruby tool, then you didn't use the same one as the Plone migrations. Unfortunately there's at least two different tools named svn2git. The one used by the Plone migrations is from the KDE community and written in Qt. Some of the more useful info is at http://techbase.kde.org/Projects/MoveToGit/UsingSvn2Git That tool doesn't really use git-svn. Instead it scan through all SVN revisions, looks at the changed paths and matches those against a ruleset. So you can tell it that both /foo/bar/trunk and /bar/trunk contain code that goes into the final bar.git repo. Since you can manually influence these rules, this tool tends to work better on projects which have moved their location in SVN a lot. It also does a single scan through the entire SVN repo and is able to generate many resulting git projects at once. Which made it perfect for a mass-migration. git-svn or tools based on it, tend to work well on small projects that always stayed in the same place, with the same structure. Hanno
On Thu, Jan 10, 2013 at 10:54 AM, Hanno Schlichting <hanno@hannosch.eu> wrote:
On Thu, Jan 10, 2013 at 3:59 PM, Jim Fulton <jim@zope.com> wrote:
On Thu, Jan 10, 2013 at 9:50 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
I did not choose to include or exclude any branch. The test migration uses the package used to migrate most Plone packages from SVN to GitHub, which uses svn2git underneath. If there's whole branches missing the migration has obviously failed.
I had a bunch of problems with ZODB and the ruby svn2git.
If you used a ruby tool, then you didn't use the same one as the Plone migrations. Unfortunately there's at least two different tools named svn2git. The one used by the Plone migrations is from the KDE community and written in Qt. Some of the more useful info is at http://techbase.kde.org/Projects/MoveToGit/UsingSvn2Git
That tool doesn't really use git-svn. Instead it scan through all SVN revisions, looks at the changed paths and matches those against a ruleset. So you can tell it that both /foo/bar/trunk and /bar/trunk contain code that goes into the final bar.git repo. Since you can manually influence these rules, this tool tends to work better on projects which have moved their location in SVN a lot. It also does a single scan through the entire SVN repo and is able to generate many resulting git projects at once. Which made it perfect for a mass-migration.
git-svn or tools based on it, tend to work well on small projects that always stayed in the same place, with the same structure.
That's interesting. Not sure what I think about it (now that I have something mostly working and don't want to figure out something else :), but definitely worth thinking about. Thanks. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
participants (10)
-
Alexandre Garel -
Gauthier Bastien -
Hanno Schlichting -
Jens Vagelpohl -
Jim Fulton -
Leonardo Rochael Almeida -
Matthew Wilkes -
Patrick Gerken -
Sebastien Douche -
Tres Seaver