[Zope] FYI: large files & linux.

[Hannu Krosing]

Ty Sarna wrote:
> 
> In article <87n1oazuao.fsf@erwin.complete.org>,
> John Goerzen  <jgoerzen@complete.org> wrote:
> > It supports it to no greater or less extent than Linux does.  The
> 
> Untrue.
> 
> > standard library defaults to using an int or a long for *seek calls.
> > The difference is semantic only, as they're both 32 bits on both
> > systems anyway.  Even if you assume unsigned (fseek is signed), that's
> > still 4gb, max.
> 
> The kernel internals of Linux use a 32 bit off_t, hence discussion of
> large file support in the next kernel release.
> 
> > I submit, therefore, that you are not seeing a Linux problem, but an
> > x86 problem :-)
> 
> I'm not seeing an x86 problem at all:
> 
> However, it turns out that for Zope use, at least with FileStorage,
> there is a problem, which turns out to be even an x86 issue but a silly
> ANSI C spec issue.  Unfortunaly Python large file support is broken on
> (32-bit CPU) NetBSD as a result -- I had to make that sparse file with a
> little C program.  Now, you could have a more than 4G BerkeleyStorage
> Zope system, since the file handling isn't done at the Python level in
> that case, but FileStorage won't work until I can work around the Python
> problem (or until NetBSD 1.5, which will fix it in a different way).
> 
> But at least it's the kind of thing that could be hacked around in an
> application by one guy in a day, rather than taking drastic kernel
> and libc changes :-)

Still it would be sensible to do it by splitting the database file (say at
1M boundaries rather than supporting some obscure non-standard system calls 
that only few platforms support. Probably also one-day hack by someone 
who knows where to start digging.

At least this is how it was solved for PostgreSQL .

-------------
Hannu