[Zope] Hardware for Zope + ZEO

[sean.upton@uniontrib.com]

On the subject of inodes, file-handles, etc.  Would it not also be wise on a
heavily loaded storage to allow for a large number of file descriptors to be
available to the ZSS process?  I think 1024 is the default under Linux for
normal users, would it not be wise just in case to run 'ulimit -HSn 8192' to
bump up the total number of file descriptors available to the ZSS and
DirectoryStorage?

One other thing that looks interesting, if there are a lot of writes, is
flash+dram combo solid-state storage devices, like those from BitMicro, to
use as an options for a DirectoryStorage journal directory.  I've never used
something like this, so I'm just speculating, but it could be possible to
decrease the likelihood of conflict errors if you are able to give quick
initial writes (that are buffered within the device dram before sync to
flash); this might safely yield better performance without needing to turn
off fsync.

Sean

-----Original Message-----
From: Mario Valente [mailto:mvalente@ruido-visual.pt]
Sent: Wednesday, January 29, 2003 9:33 AM
To: Toby Dickenson; Wankyu Choi; zope@zope.org
Subject: Re: [Zope] Hardware for Zope + ZEO


At 14:17 29-01-2003 +0000, Toby Dickenson wrote:
>On Wednesday 29 January 2003 1:37 pm, Mario Valente wrote:
>
>>    OK. Make sure that the clients are connected to the backend
>>  server (the one above) on a private network.
>
>For security or performance reasons?
>

  Both, of course.



>> fast. We've been looking at DirectoryStorage but right now its still not
>> final 
>
>FYI, we hit 1.0.0 a few weeks ago. 
>

  Yes, I'm aware of that. Nonetheless we prefer to "road test" it for
 a while on less important less traficked sites before we go whole hog :-)



>> and we're afraid that at some point (ie large sites with lots of
>> objects) the OS might run out of file handles.
>
>Apart from two lock files, DirectoryStorage only opens file handles 
>momentarily. 
>
>On some filesystems you may want to be concerned with running out of
inodes. 
>On linux I strongly recommend reiserfs, which does not have this problem 
>because it can create inodes on demand.
>

  I was actually thinking inodes when I wrote filehandles. Thanks for
 the aditional info on reiserfs.



>> All images should be stored as External
>>  Files (as well as any other BLOBs like PDF files, DOC files, etc).
>
>Im not sure that advice is universally true, unless your blobs are never 
>change or mastered elsewhere. Putting everything in ZODB/ZEO simplifies 
>backups, replication, and distribution of changes.
>

  If you put all images/docs/whatever in a single directory or directory
 tree you have simpler solutions to backup, replication and distribution.
  We found out by experience that its not beneficial to store large binary
 files in Zope, especially when there are constant file uploads going on.

  C U!

   -- Mario Valente


_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )