Re: [Zope] Large data.fs (just the Faqs)
Hi Marc, Jason,
From: "Jason C. Leach" <jleach@drivingbeat.com> Reply-To: jleach@mail.ocis.net Date: Wed, 1 Aug 2001 11:03:06 -0700 To: zope@zope.org Subject: [Zope] Large data.fs (just the Faqs).
hi,
So to have a Data.fs larger than 2GB I need:
1. To re-compile python w/ large file supprt. 2. Use the 2.4.x Linux Kernel. 3. Pray.
marc lindahl wrote:
And use an FS capable of >2GB files (like XFS)
Sorry, not quite correct ... you could use ext2 as well since its fully LFS-compliant. ext2 itself was never the problem, but (as someone had already pointed out in here) the VFS-layer in Linux (which was heavily influenced by i386-architecture -> 32bit). But you should not mix up systems with LFS (large file support) an such withpout (e.g. Linux 2.2.). The 2.2 based System will be able to read from large files, but nothing more. Perhaps you might have a look at: http://www.suse.de/~aj/linux_lfs.html for some more details (esp. in respect to the C-libs used). ta -------------------------------------------------------------------------- smartferret it-consulting Dipl.-Ing. Linzbichler KEG Dipl.-Ing. Takashi Veikko Linzbichler Tannhofweg 28/3 A-8044 Graz, Austria Tel.: 0316 / 39 89 40 -0 Fax: 0316 / 39 89 40 -20 Mobil: 0676 / 31 26 286 eMail: takashi.linzbichler@smartferret.com WWW: http://www.smartferret.com --------------------------------------------------------------------------
From: Takashi Linzbichler <takashi.linzbichler@smartferret.com>
you could use ext2 as well since its fully LFS-compliant. ext2 itself was never the problem, but (as someone had already pointed out in here) the VFS-layer in Linux (which was heavily influenced by i386-architecture -> 32bit).
Reading your reference, it looks like SuSe has added LFS recently, but I don't see that RedHat has it (checking the redhat site, only mentions that glibc supports it, and I know from experience that with the standard installation, it doesn't, with 7.0 or 7.1).
But you should not mix up systems with LFS (large file support) an such withpout (e.g. Linux 2.2.). The 2.2 based System will be able to read from large files, but nothing more.
From my point of view - I'm not going to modify the makefiles of everything and recompile it, from bash on up, so it's an academic distinction, given my platforms (i386 and ppc).
So, currently, 'out of the box', SuSe is good, I believe someone else mentioned debian as well. For redhat, you need an alternative (XFS was painless for me).
Perhaps you might have a look at:
http://www.suse.de/~aj/linux_lfs.html
for some more details (esp. in respect to the C-libs used).
On 02 Aug 2001 10:19:41 -0400, marc lindahl wrote:
From: Takashi Linzbichler <takashi.linzbichler@smartferret.com>
you could use ext2 as well since its fully LFS-compliant. ext2
itself
was never the problem, but (as someone had already pointed out in here) the VFS-layer in Linux (which was heavily influenced by i386-architecture -> 32bit).
Reading your reference, it looks like SuSe has added LFS recently, but I don't see that RedHat has it (checking the redhat site, only mentions that glibc supports it, and I know from experience that with the standard installation, it doesn't, with 7.0 or 7.1).
Not True. it may be true that Python has a problem with large files, it is not true that RH does. Though, nearly all of my test scripts ar ein python, and I do not have anu issues on any of my machines wiht >2Gb files.
But you should not mix up systems with LFS (large file support) an
such
withpout (e.g. Linux 2.2.). The 2.2 based System will be able to read from large files, but nothing more.
From my point of view - I'm not going to modify the makefiles of everything and recompile it, from bash on up, so it's an academic distinction, given my platforms (i386 and ppc).
So, currently, 'out of the box', SuSe is good, I believe someone else mentioned debian as well. For redhat, you need an alternative (XFS was painless for me).
Perhaps a little background is in order. I work for Hewlett packard, testing their Virtual Array product on Linux. I have several machines with stock Redhat 7.1, with filesizes in the several gigabyte range (and one that is nearly a terabyte!). I do this on a daily basis, creating, deleting, and building. It works. On some of them, I do not even have XFS installed. Most of these files are on ext2, bone-stock redhat 7.1, as that is the testing requirement. Technically, Redhat 6 supported LFS (*cough*on ext2*cough*)... on non-32 bit platforms. :) I have a couple of 3 gigabyte sized Data.fs files , and a much, much larger one ... how does 198GB grab you? ;). To say that RH 7.1 does not suport LFS is simply wrong. In fact, if it were true, your XFS partition would likewise have an issue with LFS. Actually, there is a limitation on filesize and filesystem size with XFS on Linux, but I doubt you'll be reaching that anytime soon. :) And it is due to the Linux VFS, as well. Sorry if it sounds like nitpicking, but it is a very important distinction. The current limit on filesize on ext2 on 32 bit systems is 2TB. Bill
From: Bill Anderson <bill@immosys.com>
Not True. it may be true that Python has a problem with large files, it is not true that RH does. Though, nearly all of my test scripts ar ein python, and I do not have anu issues on any of my machines wiht >2Gb files.
Thanks for bearing with me.... checking my notes - I did not recompile Python when checking the stock RH 7.1, so perhaps *that* was the problem I had! (testing zope with >2GB data.fs)
Perhaps a little background is in order.
I'm glad there are folks with your background, around for some 'tech support'!! best, Marc
Hi Marc, [we should keep this task short, it's etting a bit OT ..] marc lindahl wrote:
you could use ext2 as well since its fully LFS-compliant. ext2 itself was never the problem, but (as someone had already pointed out in here) the VFS-layer in Linux (which was heavily influenced by i386-architecture -> 32bit).
Reading your reference, it looks like SuSe has added LFS recently, but I don't see that RedHat has it (checking the redhat site, only mentions that glibc supports it, and I know from experience that with the standard installation, it doesn't, with 7.0 or 7.1).
OK, perhaps the reference was chosen a bit unluckily, but it was the first one at hand. It does _not_ depend on the Distro you're using, as long as you've got Kernel 2.4.x. (If you don't believe me, at least believe Bill A., who already pointed this out :-) ) Nothing to patch, recompile, or whatever. Just install and go with it.
From my point of view - I'm not going to modify the makefiles of everything and recompile it, from bash on up, so it's an academic distinction, given my platforms (i386 and ppc).
So, currently, 'out of the box', SuSe is good, I believe someone else mentioned debian as well. For redhat, you need an alternative (XFS was painless for me).
Yup and there _are_ reasons to use XFS (jounaling ...), but we shouldn't mix up the topics. ta -------------------------------------------------------------------------- smartferret it-consulting Dipl.-Ing. Linzbichler KEG Dipl.-Ing. Takashi Veikko Linzbichler Tannhofweg 28/3 A-8044 Graz, Austria Tel.: 0316 / 39 89 40 -0 Fax: 0316 / 39 89 40 -20 Mobil: 0676 / 31 26 286 eMail: takashi.linzbichler@smartferret.com WWW: http://www.smartferret.com --------------------------------------------------------------------------
participants (3)
-
Bill Anderson -
marc lindahl -
Takashi Linzbichler