[Zope] Data Insert Problems

[Jay Zeemer]

Chris,
I will look into using the SimpleUserFolder you spoke of thank you for the
recommendation.

We are currently writing to a MS SQL database.  The form we are using
contains 5 fields, and each insert causes the Data.fs to grow by approx 27K,
the final form will be 10 times this size.  If we grep the Data.fs for data
we inserted we can find lines in the file containing this information.  Do
you believe this could be a problem with the database driver we are
currently using??

We are currently running ZEO, and manually pack it during testing at about 1
to 1.5 gigs.  This usually drops it down to the 200 meg range.

We are seeing approximately 60% of the data we inserted on the front-end
make it to the SQL database.  We believe this is because of the conflict
errors we are logging at 20 concurrent users.  We need the system to support
at a minimum 34 concurrent users on this form, more would be better.  

We have also tested this on a single box with a local Data.fs and have had
similar results when testing through put on the form.  We are re-running
this test to check some other things I will let you know if the results
change.

It would appear our largest problem is with getting information moved
through the Data.fs file.  Is there any way to either work around the
Data.fs or a way to increase its ability to handle multiple writes??

Thanks for the help in this problem!!

Jay

-----Original Message-----
From: Chris Withers [mailto:chris at simplistix.co.uk]
Sent: Wednesday, March 23, 2005 9:08 AM
To: Jay Zeemer; zope at zope.org
Subject: Re: [Zope] Data Insert Problems


Hi Jay,

Jay Zeemer wrote:

> Our primary requirements for useing XUF is to allow external
> authentication of users from a database??  Is there a better solution then
> this??  

If users from a relational database is all you want, then use 
SimpleUserFolder and follow the included docs ;-)
(I have a newer unreleased version if you run into any problems, but you 
shouldn't)

> Also, the app we would like to use Zope for is a very write itensive
> application,

Yes, but writing to what?
What RDB are you using?

> the problems we are seeing with Zope currently are that the
> Data.fs is growing at a pretty decent size because the form appears to be
> writting to the Data.fs first 

What evidence do you have to support this hypothesis?

> being stored.  Our estimates have us causing the Data.fs to grow by more
> then 2 gigs per day. 

How often are you packing? Are you running ZEO?

> Is there any possible way to increase the load that
> Zope's Data.fs can handle, currently we are seeing about a 40% drop at 20
> concurrent users, over a few hours.  

40% of what? relative to what?

> Do you know of any way to ramp up the
> number of connections that Data.fs can handle??

Why do you think you need to? ;-)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk