On Wednesday 29 January 2003 8:27 pm, sean.upton@uniontrib.com wrote:
On the subject of inodes, file-handles, etc. Would it not also be wise on a heavily loaded storage to allow for a large number of file descriptors to be available to the ZSS process? I think 1024 is the default under Linux for normal users, would it not be wise just in case to run 'ulimit -HSn 8192' to bump up the total number of file descriptors available to the ZSS and DirectoryStorage?
I dont see why file descriptor usage would increase with load. ZEO server uses a file descriptor per connection, but you need hundreds or ZEO clients before this is a problem.
One other thing that looks interesting, if there are a lot of writes, is flash+dram combo solid-state storage devices, like those from BitMicro, to use as an options for a DirectoryStorage journal directory. I've never used something like this, so I'm just speculating,
Also speculating; both reiserfs and ext3 have mount options so that *all* writes go through its journal. Putting the filesystem journal on a solid state block device should increase FileStorage and DirectoryStorage burst write performance without needing any explicit implementation in those storages. (reiserfs in 2.4.20 needs a patch for this) BDB has options to relocate its journal (aka log files), which is a well known performance trick.
but it could be possible to decrease the likelihood of conflict errors if you are able to give quick initial writes (that are buffered within the device dram before sync to flash); this might safely yield better performance without needing to turn off fsync.
Interesting idea. The probability of getting a ConflictError is proportional to the total transaction time. Speeding up any part of it would help - reads, writes, and application level logic included.