DirectoryStorage and UFS2
Howdy: I'd have a Zope/Zeo site that I'd like to set up to use DirectoryStorage. Platform is FreeBSD5.2.1 using UFS2 w/softupdates. Probably will also mirror with vinum. DirectoryStorage docs state: bushy: If you are using a filesystem that is inefficient with directories containing larger than a few hundred items. This is optimal for most conventional filesystems, such as ext2 on linux. In this format each object is given its own directory nested inside 8 levels of subdirectory. Each revision of an object has its own file in that directory. chunky: If you are using a filesystem that is most efficient with directories containing tens of thousands of subdirectories, or hundreds of thousands of files. This is optimal for filesystems such as reiser3 or JFS on linux. In this format, one subdirectory is shared by up to 65000 objects. That subdirectory is nested inside 4 levels of subdirectory. I am unsure whether UFS2 would best utilize the "chunky" or "busy" options. The 64 bit pointers allow for up to 65K subdirs w/in a dir, correct? So the large number of subdirs under chunky format could be handled. Dirhash and dirpref can deal can cope with large numbers of files per directory, correct. But then doesn't UFS2 still use more linear model? Any insights from the FBSD/filesystem gurus would be appreciated. -- Best regards, Ken Gunderson GPG Key-- 9F5179FD "Freedom begins between the ears." -- Edward Abbey
Ken Gunderson wrote:
I am unsure whether UFS2 would best utilize the "chunky" or "busy" options. The 64 bit pointers allow for up to 65K subdirs w/in a dir, correct? So the large number of subdirs under chunky format could be handled. Dirhash and dirpref can deal can cope with large numbers of files per directory, correct. But then doesn't UFS2 still use more linear model?
Any insights from the FBSD/filesystem gurus would be appreciated.
We use DirectoryStorage on UFS+softupdates on FreeBSD 4.x and use bushy. I did a few simple tests of bushy vs. chunky and didn't get any significant difference. Whether this holds true for UFS2 or not I don't know. -Matt -- Matt Hamilton matth@netsight.co.uk Netsight Internet Solutions, Ltd. Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Design | Zope/Plone Development & Consulting | Co-location | Hosting
On Wednesday 19 May 2004 11:18 am, Matt Hamilton wrote:
Ken Gunderson wrote:
I am unsure whether UFS2 would best utilize the "chunky" or "busy" options. The 64 bit pointers allow for up to 65K subdirs w/in a dir, correct? So the large number of subdirs under chunky format could be handled. Dirhash and dirpref can deal can cope with large numbers of files per directory, correct. But then doesn't UFS2 still use more linear model?
Any insights from the FBSD/filesystem gurus would be appreciated.
We use DirectoryStorage on UFS+softupdates on FreeBSD 4.x and use bushy. I did a few simple tests of bushy vs. chunky and didn't get any significant difference. Whether this holds true for UFS2 or not I don't know.
FWIW-- I did a few informal tests as well. Chunky "feels" a bit faster, but I didn't test with huge dataset, so I'm not sure about scalability. Chunky also seems to require a bit less disk space, e.g. virgin dirstorage is 26MB vs 21MB. Guess I'll play it safe and stick with bushy. Thanks for the input. -- Best regards, Ken Gunderson GPG Key-- 9F5179FD "Freedom begins between the ears." -- Edward Abbey
participants (2)
-
Ken Gunderson -
Matt Hamilton