Not exactly. It's a combination of server load (long requests) and other objects that are read. I think that the transaction management of Zope have problems with request on objects that take a long time. So if a request takes too much time and another request want to access another object than it might crash.

have you tried to use the logging features of ZOPE? z2.py STUPID_FILE_LOGGER=var/Zope.log I got this information from http://www.zope.org/Members/michel/Projects/Interfaces/LOGGING.txt/ I really would like to help try to solve your problems and then we could try to bring them up to Digital Creations. ZOPE is a really decent product and the problems I've seen are people not architecting ZOPE around their problem (this is not intuitive, like some of ZOPE) but solving their problems with ZOPE (more intuitive) disregarding how ZOPE works (things should be abit more bullet proof).

No. I switched off the syncing and the problems are the same. In debug mode of zope I get error messages on console like 'conflict competing write ...'. It seems that zope have bugs rendering the DTML sites to HTML. That problem does not break every request when this message occurs.

I believe the STUPID_FILE_LOGGER will give you more information on this. I should have brought it up earlier. and we should have been discussing this on the ZOPE mailing list so we could have this archived. those conflict competing writes usually have path associated with it. are all of these occuring in the same location in the ZODB?

On Windows sometimes it breaks nothing, sometimes only this request (thread) breaks and sometimes the complete zope crashes and the process is killed or sometimes the process runs but handles no requests. We get this kind of error at least once a day (on good days) and every hour on bad days depending on load I think.

yikes, this is very bad. Andy McKay has helped port ActiveStates website to ZOPE and I wonder if they are experience the same problems.

On Linux it's no problem, because the worse case is that the thread is killed and started again. Zope is NOT completely killed or hangs. All 2-3 months the process crashes, but a little shell script restarts it and all is fine.

there is no doubt that ZOPE has more experience and is favored to run on *nix ,from what I understand.

Yes. The main point is that you should use zope's capabilities as less as possible for such big projects. I use zope nearly only for DTML and user authorization. Big Files are in filesystem, database is SQL, backend is python, some functions of zope are patched (username logging of http_server.py, my own error pages for HTML errors; e.g. 404, 401, ...). So I don't really use all zope's possibilities, but it's too slow anyway.

the big files should be served from apache and not ZOPE. that is the lesson to be learned I believe.

example: I wrote some little benchmarking tests. I do a search query with about 40 objects as result. These objects (complete classes with data and methods) are return to DTML as a list. DTML access the data to display with methods like 'getDate', 'getName', 'get....'. The backend generates these objects and sets the values with methods 'setDate', 'setName' ,.... . I benchmarked it and the result was that this search request needs 1829ms (no exact value, because of python benchmark script; I think it's a little bit faster than this values). Than I began to optimize and test some methods, functions, ... to get slow commands. I tried to avoid these complex objects, generate only simple container classes that contains only plain data. I access these values from DTML directly with obj.date, obj.name, obj.xxx. I left all methods and complex things out. I read MIME type information on download and not at file object generating time. I left out my python function 'quoteForHTML' that do the HTML Quoting for all special chars (I will write a python C module these days if I find no other way). And the result was for the same request it took only 400ms. So I have to optimize all my code to get it work. That's not simple. The code is about 11500 lines. And the object oriented handling is thrown away for DTML requests because of speed problems. Python and Zope are not fast enough for that complex backend. This code in PHP would be faster and does not end in crashes. PHP code is so ugly that I don't want it, but it is stable.

You say the code is 11500 lines long. You are using German characters I assume, I wonder if this is causing any problem? there is a german ZOPE company beehive that you may want to get in touch with. they have some extensive experience and may be able to out, http://www.beehive.de/index.html . this is why I would like to see your problem solved. I have used PHP in the past and am not a big fan. It is stable, fast, and has a larger community. How experience with Python are you? I see you had setName getName, in python you usually directly set/get attributes with instance.property=value and value=instance.property, overloading __setattr__ and __getattr__ if you wanted to do a tiny more than set/get values as well.

Who are you exactly ? It seems that you know a lot about zope. Are you developing on zope ? I read you brought up ZEO, right ?

I'm just some programmer in Texas, USA. I have been reading and playing with ZOPE for almost two years. I am not developing on ZOPE, but may start in the next few months (I just lost my job this week). I have not done anything with ZEO besides getting it up and running in a test environment -- nothing production such as your case. I'm CCing the ZOPE list. ~runyaga