[Zope] please help with membership design and DCOracle

[paulabrams@yahoo.com]

Hi guys,


We are trying to build a web application that needs to have _many_
members (kind of like a portal).  We need a way to store large numbers
of member profiles, each with their own username password, and a way to
track usage statistics.

Zope appears to be very well suited for caching and displaying content
on the site, and we have built most of the application so far with
relatively little hassle.  This is our first project using Zope and we
like it a lot.  kudos!  

There is a reason for this post, however :)  We need some help
validating our approach for handling members and some help with a
couple of specific problems. Please help!


MEMBERS
The next step is to add members, which requires large numbers of writes
to the system, particularly since the data generated by the application
(~200K of HTML per usage) must be saved. I am looking at two approaches
and am desperately wondering if anyone has an opinion.  

A) Store all member data as folders and properties in Zope. 
B) Use a RDBMS for member data storage

Assumptions: (please correct me if I am wrong about any of these)
I.   Zope can select cached data faster than a RDBMS.
II.  A RDBMS can handle frequent and/or large writes of data faster
than Zope.

It seems like option B is a better solution because it maximizes Zopes
strengths (RAD, displaying cached static content in a dynamic way) and
minimizes Zopes weaknesss (slow writes). In fact, we have successfully
developed a large portion of the membership services using this
approach with Oracle.


PROBLEMS  

[ Environment:  Zope 1.10.3, Linux Redhat 6,  Oracle 8,  DCOracle 2.0.0
]

1. One big problem we are currently running into involves member
authentication. If we are going to avoid writing membership data into
Zope, that means usernames & passwords need to be stored in the RDBMS.
We can get this information from the user and authenticate him/her, but
we then need a way to assign him/her a role so that we can use standard
Zope security to protect our site.

2. Writing large amounts of data into the database seems to be a
problem as well. We are flexible about the datatype used, so long as it
can hold ~100K of html, but are having problems with the
implementation. We tried using a LONG datatype, and then trying to
store the html using a ZSQL method containing the following code
snippets (in separate tests):
a) BODY='<!--#var body sql_quote-->'
b) BODY=<!--#sqlvar body type=string-->
We get back an oracle error "string literal too long" if the data
exceeds 4000 characters. Is there another way to write this that will
work?


Thanks in advance,
-Paul