[Zope-CMF] best practices - dealing with large files and avoiding data corruption

[Sam Brauer]

I have two problems that may or may not be related, but I'll go ahead 
and ask about both of them...

I expect other folks with CMF sites have run into the problem of dealing 
with users that need to upload large files (by "large" I mean 5-100Mb) 
into a CMF site.  I'm wondering how other people have dealt with this 
sort of situation.

Yesterday one of our Zope sites "locked up" the server (had to be rebooted).
I think a contributing factor may have been that someone was uploading 
some largish files into the site.  Unfortunately, there's no telling 
error message in the log.

I've seen the CMFExtFile product, but it doesn't look to be supported 
anymore and the project page for it says that it doesn't work with Zope 
2.5 and higher.

I'm also thinking about using DirectoryStorage instead of the normal 
FileStorage.   Not sure if it deals better with large files or not, but 
I'm hoping that perhaps it is more resistent to corruption than the 
FileStorage.

After only a couple of weeks with CMF sites in production, I've already 
seen 2 corrupt data errors.  Not to imply that CMF was responsible for 
the errors, it's just that the only Zope sites I have are CMF sites.

The corruption problems are really unsettling to me.
At least I was able to recover (using the fsrecover.py from 
Zope-2.7.0-a1), but shutting down zope and running that utility means 
downtime for all sites served from that Zope instance.

I love Zope, but need it to be reliable.  I'm running Zope 2.6.1 and 
have seen that there have been several ZODB fixes since then, but 2.6.1 
is still the latest stable Zope release.

Have any other folks had similar experiences?
Any suggestions for avoiding data corruption?
What about dealing with large file uploads?

(Sorry if these are totally unrelated issues, but I'm curious about 
both... so don't be shy about responding to just one of these issues.)

Thanks for any help,
Sam