Howdy! Has anybody tried to use R to create graphics which then get delivered by Zope? Ragnar
"RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes:
RB> Howdy! Has anybody tried to use R to create graphics which RB> then get delivered by Zope? If by R, you mean www.r-project.org (the R statistical package), I know some people working on it, via Zope integration of "R scripts" using the RSPython interface. I can comment more if this is what you mean. If you mean something else, I can't help. best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
Yep, that's exactly what I mean. It would be very nice if you could comment some more. Ragnar On 04 Feb 2002 06:32:58 -0800 rossini@blindglobe.net (A.J. Rossini) wrote:
RB> Howdy! Has anybody tried to use R to create graphics which RB> then get delivered by Zope?
If by R, you mean www.r-project.org (the R statistical package), I know some people working on it, via Zope integration of "R scripts" using the RSPython interface. I can comment more if this is what you mean. If you mean something else, I can't help.
best, -tony
-- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
"RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes:
RB> Yep, that's exactly what I mean. It would be very nice if you RB> could comment some more. So, I (and another group, independently) have discussed integration, with the idea of having Zope be a WWW-front end for statistical analysis, both report generation as well as interactive style. There are pretty much 3 approaches that seem to work, and note that as is usual, coolness/slickness factor is inversely proportional to how well it works. Version 1: treat R as batch. This is ugly, and works (i.e. shell-out in CGI-style, constructing image files for the graphics and use LocalFS to read them back in. Version 2: Same, but use RSPython (www.omegahat.org) to embed R within Zope. Eliminates the CGI scripting, but you still have either remote files and/or LocalFS stuff to deal with. The first catch is that RSPython is borderline stable. It doesn't recover from mistakes (like missing objects) very well. Version 3: ZopeR/RZope. It's under development, and basically adds an "RScript" object, as well as allowing Zope and R to read each others data (i.e. R to use ZODB for writing). It's the coolest, least stable, and most vapor of the approaches. It uses RSPython for integration. What I'd like to do is CMF-ify a stable ZopeR/RZope, for use with WWW-based instruction as well as "analysis deliverables" to stat consulting clients (and some labs I work with). It's made interesting by the ease of retrieving/constructing bioinformatics datasets using things like KebasData and other "reformatters/fetchers". Timetable on #3 is probably "1-2 years after whenever funding appears", though there is a large commercial entity considering work on it in-house as a possible basis for an integrated enterprise-level (bio/clinical/medical) informatics system. best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
On Mon, 2002-02-04 at 15:42, A.J. Rossini wrote:
"RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes:
RB> Yep, that's exactly what I mean. It would be very nice if you RB> could comment some more.
So, I (and another group, independently) have discussed integration, with the idea of having Zope be a WWW-front end for statistical analysis, both report generation as well as interactive style. <snip>
Version 1: treat R as batch. This is ugly, and works (i.e. shell-out in CGI-style, constructing image files for the graphics and use LocalFS to read them back in.
What's wrong with that approach? It doesn't have to be ugly. You can distribute the work to other computers, the batch will run as a separate process which gives you a certain number of advantages, and *it works*. I use imagemagick like this and I'm very happy with it. The only issue is writing x-platform code can be a bit tricksy. seb
"sb" == seb bacon <seb@jamkit.com> writes:
sb> On Mon, 2002-02-04 at 15:42, A.J. Rossini wrote: >> >>>>> "RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes: >> RB> Yep, that's exactly what I mean. It would be very nice if you RB> could comment some more. >> So, I (and another group, independently) have discussed >> integration, with the idea of having Zope be a WWW-front end >> for statistical analysis, both report generation as well as >> interactive style. sb> <snip> >> Version 1: treat R as batch. This is ugly, and works >> (i.e. shell-out in CGI-style, constructing image files for the >> graphics and use LocalFS to read them back in. sb> What's wrong with that approach? It doesn't have to be ugly. sb> You can distribute the work to other computers, the batch will sb> run as a separate process which gives you a certain number of sb> advantages, and *it works*. I use imagemagick like this and sb> I'm very happy with it. The only issue is writing x-platform sb> code can be a bit tricksy. Nothing is wrong with this approach. As I stated, it's ugly and works. I've used variants on it (using R) since 1996. Of course, beauty is in the eye of the beholder. Since my life isn't based on writing functional software (though that's an important component), I've got a different aesthetic scale. There happen to be more elegant but much less stable approaches. The critical point is that R was designed for interactive (statistical) data analysis, which is what I mostly care about. The above is fine for static scripts (not necessarily static results, since databases can change), however, I'm much more interested more in a discussion-style flexible UI for statistical analysis than an "embed results within an object by reading from the updated database". The CGI approach is just not terribly useful for building an effective UI for data analysis scripts, though (by effective, I mean "the average statistician or student can use it with minimal instructions", not web programmers who are comfortable with semi-arcane tools). It doesn't enhance rapid construction based on general principles. Been there, done that, need to move on. Note that by CGI approach, I'm thinking of the crude "R or other image processing programs as batch" appoach. Of course, we can also think of Zope as a CGI approach unto itself. best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
Thanks a lot! RSPython seems to be the most likely candidate for my purposes. I actually don't need R's interactive features in this case. What I need is a Python script that accesses an SQL database, unpickles data that is stored in a blob, and then generates an image (a rather large GIF) with a predefined style or calculates a couple of coefficients. Generating images seems to be the hardest part. How are you handling temporary files with more than one concurrent thread and possible security issues? I've never used temporary files so far. Cheers, Ragnar
So, I (and another group, independently) have discussed integration, with the idea of having Zope be a WWW-front end for statistical analysis, both report generation as well as interactive style.
There are pretty much 3 approaches that seem to work, and note that as is usual, coolness/slickness factor is inversely proportional to how well it works.
Version 1: treat R as batch. This is ugly, and works (i.e. shell-out in CGI-style, constructing image files for the graphics and use LocalFS to read them back in.
Version 2: Same, but use RSPython (www.omegahat.org) to embed R within Zope. Eliminates the CGI scripting, but you still have either remote files and/or LocalFS stuff to deal with. The first catch is that RSPython is borderline stable. It doesn't recover from mistakes (like missing objects) very well.
Version 3: ZopeR/RZope. It's under development, and basically adds an "RScript" object, as well as allowing Zope and R to read each others data (i.e. R to use ZODB for writing). It's the coolest, least stable, and most vapor of the approaches. It uses RSPython for integration.
What I'd like to do is CMF-ify a stable ZopeR/RZope, for use with WWW-based instruction as well as "analysis deliverables" to stat consulting clients (and some labs I work with).
It's made interesting by the ease of retrieving/constructing bioinformatics datasets using things like KebasData and other "reformatters/fetchers".
Timetable on #3 is probably "1-2 years after whenever funding appears", though there is a large commercial entity considering work on it in-house as a possible basis for an integrated enterprise-level (bio/clinical/medical) informatics system.
best, -tony
-- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
"RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes:
RB> Thanks a lot! RSPython seems to be the most likely candidate RB> for my purposes. I actually don't need R's interactive RB> features in this case. What I need is a Python script that RB> accesses an SQL database, unpickles data that is stored in a RB> blob, and then generates an image (a rather large GIF) with a RB> predefined style or calculates a couple of coefficients. RB> Generating images seems to be the hardest part. How are you RB> handling temporary files with more than one concurrent thread RB> and possible security issues? I've never used temporary files RB> so far. If you want this to really work, I'd go one further than Seb's "critical" comments (remember, I'm an academic, I can afford crashes, especially of the "crash early, crash often" in research projects :-), and suggest using Python, PIL, or GDchart for this, NOT R. Really, "right tools for the job", "stability", and all that stuff. Instead, for numerics, I'd suggest NumPy or SciPy (both excellent, though lacking some (all!) of S/R's better features for data analysis and annotation), for the computations. I've done just as well with Python's SQL methods as I've done with R's (maybe better), and there are a number of Zope products for doing what you are proposing. Python seems to be a better choice -- I've been doing Flow Cytometry analysis in both R and Python (flow cyt files are like the files you are describing) and Python definitely has better behaviour (not to mention that you'll have a 1-2 second start-up cost server-side for getting R "fired up"). Now if you are looking at incorporating R for a "lab workbench/notebook" style project, that would be a another story... best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
On 05 Feb 2002 05:31:41 -0800 rossini@blindglobe.net (A.J. Rossini) wrote:
"RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes:
RB> Thanks a lot! RSPython seems to be the most likely candidate RB> for my purposes. I actually don't need R's interactive RB> features in this case. What I need is a Python script that RB> accesses an SQL database, unpickles data that is stored in a RB> blob, and then generates an image (a rather large GIF) with a RB> predefined style or calculates a couple of coefficients. RB> Generating images seems to be the hardest part. How are you RB> handling temporary files with more than one concurrent thread RB> and possible security issues? I've never used temporary files RB> so far.
If you want this to really work, I'd go one further than Seb's "critical" comments (remember, I'm an academic, I can afford crashes, especially of the "crash early, crash often" in research projects :-), and suggest using Python, PIL, or GDchart for this, NOT R. Really, "right tools for the job", "stability", and all that stuff.
Instead, for numerics, I'd suggest NumPy or SciPy (both excellent, though lacking some (all!) of S/R's better features for data analysis and annotation), for the computations.
I've done just as well with Python's SQL methods as I've done with R's (maybe better), and there are a number of Zope products for doing what you are proposing.
Python seems to be a better choice -- I've been doing Flow Cytometry analysis in both R and Python (flow cyt files are like the files you are describing) and Python definitely has better behaviour (not to mention that you'll have a 1-2 second start-up cost server-side for getting R "fired up").
Now if you are looking at incorporating R for a "lab workbench/notebook" style project, that would be a another story...
Hmm, do you mean RSPython shouldn't be used in a production environment because RSPython *itself* isn't ready for production or would it rather be *my* code that'd crash for trial-and-error reasons? Cheers, Ragnar
"RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes:
RB> Hmm, do you mean RSPython shouldn't be used in a production RB> environment because RSPython *itself* isn't ready for RB> production or would it rather be *my* code that'd crash for RB> trial-and-error reasons? I know nothing about your coding, but I'm intimately familiar with the internal guts of RSPython. I'd predict the former, no matter how good the latter is :-). best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
On 05 Feb 2002 08:56:50 -0800 rossini@blindglobe.net (A.J. Rossini) wrote:
RB> Hmm, do you mean RSPython shouldn't be used in a production RB> environment because RSPython *itself* isn't ready for RB> production or would it rather be *my* code that'd crash for RB> trial-and-error reasons?
I know nothing about your coding, but I'm intimately familiar with the internal guts of RSPython. I'd predict the former, no matter how good the latter is :-).
Heehee :) So it seems like I'll have to take a closer look at GDChart. In the future I'll probably have to generate PDF reports containing EPS graphics but I can always have a cron job running that generates the graphics etc. and then serve that stuff from the filesystem. Cheers, Ragnar
"RB" == Ragnar Beer <rbeer@uni-goettingen.de> writes:
RB> Hmm, do you mean RSPython shouldn't be used in a production RB> environment because RSPython *itself* isn't ready for RB> production or would it rather be *my* code that'd crash for RB> trial-and-error reasons? >> I know nothing about your coding, but I'm intimately familiar >> with the internal guts of RSPython. I'd predict the former, no >> matter how good the latter is :-). RB> Heehee :) So it seems like I'll have to take a closer look at RB> GDChart. In the future I'll probably have to generate PDF RB> reports containing EPS graphics but I can always have a cron RB> job running that generates the graphics etc. and then serve RB> that stuff from the filesystem. Actually, you probably can do that all on the fly in Zope (since you could use png instead of eps for PDF docs, though I'm sure that using one of the image conversion tools linked through Python (i.e. ImageMagick has python bindings, I thought?) or native python tools would work. best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ ---------------- FHCRC: M: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX --- I'm 40% time until March 1st. Try email the other 3 days.... -----
RB> Hmm, do you mean RSPython shouldn't be used in a production RB> environment because RSPython *itself* isn't ready for RB> production or would it rather be *my* code that'd crash for RB> trial-and-error reasons?
>> I know nothing about your coding, but I'm intimately familiar >> with the internal guts of RSPython. I'd predict the former, no >> matter how good the latter is :-).
RB> Heehee :) So it seems like I'll have to take a closer look at RB> GDChart. In the future I'll probably have to generate PDF RB> reports containing EPS graphics but I can always have a cron RB> job running that generates the graphics etc. and then serve RB> that stuff from the filesystem.
Actually, you probably can do that all on the fly in Zope (since you could use png instead of eps for PDF docs, though I'm sure that using one of the image conversion tools linked through Python (i.e. ImageMagick has python bindings, I thought?) or native python tools would work.
You mean GDChart can also produce vector images? That would be cool! GDChart's documentation is a bit shortish ... Perhaps I should take the painful way and look a the source. Cheers, Ragnar
On Wed, Feb 06, 2002 at 04:09:35PM +0100, Ragnar Beer wrote:
You mean GDChart can also produce vector images? That would be cool! GDChart's documentation is a bit shortish ... Perhaps I should take the painful way and look a the source.
Reportlab can generate PDF, postscript, and bitmaps from text, vector graphics, bitmap images, whatever ... it's open-source and it's python, too. Demo at http://www.reportlab.com/cgi-bin/graphics_demo.cgi Note that the bitmap rendering option needs to be downloaded separately at the "addons" section of the site. Another nice vector-graphics option for python is sping, formerly known as piddle: http://piddle.sourceforge.net -- paul winkler home: http://www.slinkp.com music: http://www.reacharms.com calendars: http://www.calendargalaxy.com
On Wed, Feb 06, 2002 at 03:01:04PM -0500, Paul Winkler wrote:
On Wed, Feb 06, 2002 at 04:09:35PM +0100, Ragnar Beer wrote:
You mean GDChart can also produce vector images? That would be cool! GDChart's documentation is a bit shortish ... Perhaps I should take the painful way and look a the source.
Reportlab can generate PDF, postscript, and bitmaps from text, vector graphics, bitmap images, whatever ... it's open-source and it's python, too. Demo at http://www.reportlab.com/cgi-bin/graphics_demo.cgi
Note that the bitmap rendering option needs to be downloaded separately at the "addons" section of the site.
Another nice vector-graphics option for python is sping, formerly known as piddle: http://piddle.sourceforge.net
And another option is ploticus (note this is NOT plplot) ploticus.sourceforge.net. Can produce png, jpeg, SVG, and postscript. Especially well suited for repetitive tasks, as it uses a non-procedural template language to define charts, and the data can be sourced from a file or standard input. Not python, but easy to invoke via os.system. Very good and extensive documentation. jim penny
participants (5)
-
Jim Penny -
Paul Winkler -
Ragnar Beer -
rossini＠blindglobe.net -
seb bacon