[Zope] Hardware for Zope + ZEO

[Toby Dickenson]

On Wednesday 29 January 2003 8:27 pm, sean.upton@uniontrib.com wrote:
> On the subject of inodes, file-handles, etc.  Would it not also be wise=
 on
> a heavily loaded storage to allow for a large number of file descriptor=
s to
> be available to the ZSS process?  I think 1024 is the default under Lin=
ux
> for normal users, would it not be wise just in case to run 'ulimit -HSn
> 8192' to bump up the total number of file descriptors available to the =
ZSS
> and DirectoryStorage?

I dont see why file descriptor usage would increase with load. ZEO server=
 uses=20
a file descriptor per connection, but you need hundreds or ZEO clients be=
fore=20
this is a problem.

> One other thing that looks interesting, if there are a lot of writes, i=
s
> flash+dram combo solid-state storage devices, like those from BitMicro,=
 to
> use as an options for a DirectoryStorage journal directory.  I've never
> used something like this, so I'm just speculating,

Also speculating; both reiserfs and ext3 have mount options so that *all*=
=20
writes go through its journal. Putting the filesystem journal on a solid=20
state block device should increase FileStorage and DirectoryStorage burst=
=20
write performance without needing any explicit implementation in those=20
storages.

(reiserfs in 2.4.20 needs a patch for this)

BDB has options to relocate its journal (aka log files), which is a well =
known=20
performance trick.

> but it could be possible
> to decrease the likelihood of conflict errors if you are able to give q=
uick
> initial writes (that are buffered within the device dram before sync to
> flash); this might safely yield better performance without needing to t=
urn
> off fsync.

Interesting idea.=20

The probability of getting a ConflictError is proportional to the total=20
transaction time. Speeding up any part of it would help - reads, writes, =
and=20
application level logic included.