Hi, I also looked at Hudson a few days ago and found it to be awesome. :) On 03/04/2010 10:10 PM, Tres Seaver wrote:
After successfully configuring the Hudson[1] continuous integration server yesterday to test the various repoze.* packages[2], I thought I would experiment with using it to drive the compatibility tests for ZTK and zopeapp. Here is what I did:
In a shell on my server:
$ wget http://hudson-ci.org/latest/hudson.war $ java -jar hudson.war --httpPort=19999
That's what blew me off my chair. :)
....
In the browser, visit 'http://laguna.palladion.com:19999, and configure the server:
- Enable security - allow signup - using Matrix - set up 'tseaver' as having all permissions - set up 'anonymous' as having 'Read' permission overall and for jobs
- Register as 'tseaver' - Turn off signup (no, I don't want you guys running arbitrary shell scripts on my server ;).
- Create a job, 'ztk-trunk-py26':
See below
http://laguna.palladion.com:19999/job/ztk-trunk-py26/
- Use SVN, pointed at 'svn://svn.zope.org/repos/main/zopetoolkit/trunk'
- Set to poll SVN every 15 minutes.
- Enable using 'svn up'
- Added build step as a shell script::
cd trunk /opt/Python-2.6.4/bin/python bootstrap.py bin/buildout bin/test-ztk bin/test-zopeapp
- Ran a build manually. The first one barfed due to a typo in my script, but the second one ran, taking 21 minutes.
Right. That was my experience as well. Also a good thing is that you can now take those as templates, or - if you like - generate those XML files into the filesystem for new builds and ping the server to reload those.
The test runs successfully to completion, but doesn't produce statistics in a form that Hudson understands. The following changes to 'zope.testing.testrunner' would make the output more informative:
- Add a flag like the '--with-xunit' flag to the Nose testrunner, which dumped results into a JUnit-compatible XML file[3].
- Add a flag like the '--with-xcompat' flag to the Nose testrunner (when run with the 'nosexcover' plugin), producing a Cobertura- compabtible XML file[4].
Right. However Hudson does understand failures/successes at least. I made a job for the ZODB where the individual steps (checkout, bootstrap/buildout, test) where separated steps so that made the UI integration a little bit better.
At that point, the build process would be nicely automated, with browsable results including coverage, pretty graphs, etc. See the 'repoze' jobs for examples of those outputs.
Here's another awesome part: You can configure matrix build jobs which would allow us to define a single job for i.e. ZODB and then say "we want this tun run in the combinations of Unix/Windows, 32/64, Python 2.4/2.5/2.6, trunk/branchx /branchy" and it goes off automatically trying to find build nodes that can provide all combinations. It took my about 10 minutes to get that going including the installment of the Windows builder.
Questions for discussion:
- I find the prettier interface and easier setup than buildbot worth running a 3rd-party Java app (rather than a 3rd party Python app). Would that be acceptable among the folks running our automated tests?
It would be for me. Right now I'll continue to clean up what we have with buildbot. I really don't want to put anybody off. Getting more coverage (however) and making it more visible and regular is my primary goal. I don't think we have to through anything out. I'll be happy to provide even more tests using other build systems in parallel.
- Is it worth adding the XML support for test results and coverage to 'zope.testing.testrunner'?
Maybe. I'm happy hacking on the testrunner. :)
- Would people be willing to run Hudson "permanently" on enough hosts to make this a reasonable replacement for buildbot (we need Windows, too)?
I'm definitely willing to run it for 32/64 bit Linux machines. I'm short on Windows licenses but Hudson seems easy to integrate securely distributed over multiples sites and easy enough to set up with the JNRE/service installer.
- Hudson seems general purpose enough to allow automating other code- related tasks, e.g., production of dependency graphs. Maybe we could even automate building Windows installers?
How awesome that would be. :)
I will try to leave the ZTK Hudson server up for a little bit, to allow folks to see what it can deliver already.
[3] http://old.nabble.com/schema-for-junit-xml-output-td22193385.html
Thanks a lot for the effort. I'm really excited about Hudson and I'm willing to help out here. Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development