On the subject of inodes, file-handles, etc. Would it not also be wise on a heavily loaded storage to allow for a large number of file descriptors to be available to the ZSS process? I think 1024 is the default under Linux for normal users, would it not be wise just in case to run 'ulimit -HSn 8192' to bump up the total number of file descriptors available to the ZSS and DirectoryStorage? One other thing that looks interesting, if there are a lot of writes, is flash+dram combo solid-state storage devices, like those from BitMicro, to use as an options for a DirectoryStorage journal directory. I've never used something like this, so I'm just speculating, but it could be possible to decrease the likelihood of conflict errors if you are able to give quick initial writes (that are buffered within the device dram before sync to flash); this might safely yield better performance without needing to turn off fsync. Sean -----Original Message----- From: Mario Valente [mailto:mvalente@ruido-visual.pt] Sent: Wednesday, January 29, 2003 9:33 AM To: Toby Dickenson; Wankyu Choi; zope@zope.org Subject: Re: [Zope] Hardware for Zope + ZEO At 14:17 29-01-2003 +0000, Toby Dickenson wrote:
On Wednesday 29 January 2003 1:37 pm, Mario Valente wrote:
OK. Make sure that the clients are connected to the backend server (the one above) on a private network.
For security or performance reasons?
Both, of course.
fast. We've been looking at DirectoryStorage but right now its still not final
FYI, we hit 1.0.0 a few weeks ago.
Yes, I'm aware of that. Nonetheless we prefer to "road test" it for a while on less important less traficked sites before we go whole hog :-)
and we're afraid that at some point (ie large sites with lots of objects) the OS might run out of file handles.
Apart from two lock files, DirectoryStorage only opens file handles momentarily.
On some filesystems you may want to be concerned with running out of inodes. On linux I strongly recommend reiserfs, which does not have this problem because it can create inodes on demand.
I was actually thinking inodes when I wrote filehandles. Thanks for the aditional info on reiserfs.
All images should be stored as External Files (as well as any other BLOBs like PDF files, DOC files, etc).
Im not sure that advice is universally true, unless your blobs are never change or mastered elsewhere. Putting everything in ZODB/ZEO simplifies backups, replication, and distribution of changes.
If you put all images/docs/whatever in a single directory or directory tree you have simpler solutions to backup, replication and distribution. We found out by experience that its not beneficial to store large binary files in Zope, especially when there are constant file uploads going on. C U! -- Mario Valente _______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )