[Zope] Some tough questions about zope... (long)

[Paul Everitt]

Tony wrote:
> This is quite a long email where I ask a lot of questions about Zope, 
> but I hope you'll be lenient with me as I think I have a few hard 
> decisions to make!

I appreciate your patience and taking the time to submit well-thought
questions.
 
> Some background:
> I'm one of the web developers here, with a remit to implement a 
> curriculum database that can;
> (a) support the Medical Faculty here
> (b) get rolled out to three other Medical-related sites, each with 
> their own 'look and feel'
> (c) possibly implement a University-wide curriculum database.

Sounds fantastic!
 
> The curriculum database is essentially delivering online study guides 
> (over 100 for each Faculty) whose source is originally from very well 
> marked up Word Documents. In addition, we include multiple choice 

This is really weird, it just reminded me of a (rather pathetic) dream I
had last night.  I dreamt that someone created a Python module that
would read Word files.

My DreamTV must be picking up the wrong channel.

> questions, file upload of lecture notes etc. and some 'active' web 
> pages showing dynamical processes in medicine etc. The curriculum 
> database needs to be searchable across its entire breadth (although 
> initially restricted to the one site), so that students and faculty 
> can find occurences of, say, 'lung cancer' in lecture resources - 
> this is to plan for curriculum management, so that we can find out if 
> we're pushing subjects too much (or indeed, whether we're pushing a 
> subject at all!). We have other applications (nearly all of which are 
> written in Perl/PHP3) and want to seemlessly integrate them into the 
> curriculum database.
> 
> Currently we do all our development work on Unix, using Apache 
> virtual web servers, MySQL as our database engine and Perl running as 
> a CGI for most of our applications. My input has been to introduce 
> PHP-3 (an Apache module with a bulit-in XML parser) and some XML 
> knowledge to the group. I've been looking at a scenario where our 
> study guides are converted in XML using Omnimark and held as static 
> XML documents in a database. They are then rendered into HTML/RTF and 
> put into static web directories. The student can run a program that 
> extracts elements from selected XML documents and creates a 
> customised set of HTML pages or RTF document that is built for them 
> 'on the fly'. My manager was a bit worried about all these static 
> pages lying about and to be honest so am I, but extracting the 
> relevant XML from a database to render into individual HTML pages 
> seems a massive drain on the system.

I've been awfully interested in XML in the last few days.  Can you
describe what user-visible benefits PHP delivers due to its integrated
XML parser?

Additionally, we are working on Zope searching for a consulting project
right now.  Actually it is Zope Cataloging.  It's still a
consulting-only option, but plans are that eventually it will go "below
the line".
 
> I found out about Zope Monday morning, dl'ed it, installed it, got 
> one or two things working... and have been worried sick ever since.

Ahhh, you too, eh?  Did you have the dream about the chimp and the
External Method where...oh, nevermind. :^)
 
> My problem? is that I can see how Zope can do so much for us, but it 
> seems to need a hell of a gut-wrench to do so.

I *think* I have some good news to report on this, but before I squawk,
could you detail this gut-wrench a bit more?

> There also doesn't seem to be an easy 'half-way' house to 
> move to Zope.
> I've read a lot of the documentation, and am doing so constantly, but 
> the wisdom of the list would be very helpful indeed.
> 
> The questions:
> a) Is there a way for us to 'dump' the non-dynamic parts of a Zope 
> database into static HTML pages. This would act as a way to say to 
> the team "look, we have a parachute!".

There have been a lot of suggestions on this one.  Here's another one: a
command-line program that takes everything out, such as the wget
program.

> b) Can I integrate with the Perl/PHP3 programs that we have (the Perl 
> programs are mainly .cgi, but the PHP3 programs are server-side 
> scripted, and as such are embedded into the HTML itself). This is so 
> that I can
> c) Is there any way of using the XML documents that I'm creating into 
> the database and parsing them to produce either HTML or RTF output?

I'd like to have this as well.  I've been doing a lot of thinking about
this lately.  It is a bit more complicated than expected, though...

> d) I know that this a terrible question to ask... but is Zope likely 
> to 'remain' open-source (we're worried about getting stuck with a 
> proprietary system)?.

This is the main reason I replied.

Having Zope as a free and Open Source(tm) platform is the heart of our
business plan.  A closed Zope would basically mean we start over
completely as a different company.  Thus, closing Zope is roughly
equivalent to closing our company.

With _that_ said, we still need to provide some assurances that we won't
legislate the connection between Zope and DC.  First, we are trying to
keep some arm's length distance between zope.org and Digital Creations.
We hope to formalize the way the community participates in the platform.
We expect to make it lucrative for other businesses to participate.

At the same time, we intend to pursue commercial opportunities atop the
platform for vertical applications.

Ultimately this is the same question put to Netscape regarding Mozilla
during the AOL talks.  The answer is simple: "The genie is out of the
bottle."

Since it is open sourced, Digital Creations must *earn* its stewardship
of Zope.  If we don't we'll become less relevant.

Earning the community's enthusiasm is pretty simple, I think.
Unfortunately we aren't doing a good job right now, though we are trying
damn hard.  But we'll keep trying...

In the meantime, I'm absolutely ecstatic to see folks in the community
step forward over the last month and lead the way.

> e) Can I 'point' an Apache virtual server to run a Zope cgi (ie 
> dropping the :port requirment).

Yep, the zope.org machine has five virtual hosts.

> f) I'd like to have the option to have user authentication 
> information coming from a database - is this possible?

Previously answered.

> g) We have something like 200 main level directories holding the 
> study guides, each with up to 200 files in them - is that pushing the 
> Zope database too far?

Ditto.

> h) We also need resources such as QuickTime movies, images PowerPoint 
> slides etc. to be uploaded - would they sit in the database or can 
> they be held somewhere else (and what happens about file name 
> conflicts)?

We put them in the database.  The 3 Mb Zope binaries are served out of
the database, the PDF files are served out of the database.  This works
so well I don't even think about it any more.

--Paul