[Zope] ZODB and large data sets

[Jimmie Houchin]

At 08:04 PM 5/24/99 GMT, Ty Sarna wrote:
>In article <3.0.5.32.19990524102307.00822100@texoma.net>,
>Jimmie Houchin  <jhouchin@texoma.net> wrote:
>> At 10:48 AM 5/24/99 -0400, Jim Fulton wrote:
>> >
>> >The file format used by the ZODB 3 FileStorage will also support 
>> >very large files on systems with large file support (as described in
>> >http://www.python.org/doc/current/lib/posix-large-files.html#l2h-1441).
>> 
>> At the above URL it mentions these OSes/systems as being capable:
>> AIX, HPUX, Irix and Solaris
>> 
>> Do you know if RedHat Linux running on an AlphaServer such as DS10 would be
>> capable?
>
>If you're looking for an open source OS for a server with bigtime
>storage requirements and want to avoid silly limits like 2GB files and
>127M swap partitions (though maybe that one was fixed in 2.2?), you
>might want to look at NetBSD. 
>
>NetBSD has had 64 bit off_t's since 1.0 (1994) on both 32 and 64bit
>platforms.  I know of a NetBSD box at NASA Ames with a ~155GB
>filesystem.  I think they had one at ~600GB for a while, before
>splitting into multiple smaller ones up for testing.  You can store a
>single 155GB file on it, if you want (though at the 30MB/s that the
>storage it's on gets, it would take you and hour and a half to write
>it!).  That doesn't even hit the tripple-indirect-block code (at
>~256TB), which has been tested (with sparse files) to work fine.
>
>While I was checking the details on this, someone else chimed in that
>he'd played with 18GB files on his machine, and someone made a 21G file
>on the spot just for fun :-)
>
>The other BSD's should at least theoretically support this as well, but
>I know it's actually been used on NetBSD.

Thanks,

That's great information.

I was really hope to build my server out of reasonably common hardware so
as to keep costs down. I also was hoping to avoid proprietary hardware and
software. I will look into the various BSDs. BSD is not nearly as well
supported on the Mac outside of OS X. I use the Mac, LinuxPPC, for
development.

It would be nice if Linux would patch this tho'.

Jimmie Houchin