Thats just a matter of trust to me. Ext2 has proven to be stable even if I did nasty things to it. Ext2 and the tools it comes with has had a lot more exposure, and a lot more time to mature (and besides Data.fs sits on top of Ext2, so you have the vulnerabilities of both, in a single point of failure). The added features of ZODB sure would be handy sometimes, and yes, the added bookkeeping is painful, but I simply wouldn't want big amounts of data in ZODB. Or in a RDBMS, for that matter. Use RDBMS or ZODB for metadata, keep the bulk in lowtech storage... ru, peter. On Thu, 3 May 2001, marc lindahl wrote:
From: Bill Anderson <bill@libc.org>
Which is worse: A corrupted Data.fs, or a corrupted .mp3 file? If you get a corruption in the backup process (Yes, it ODES happen), or the restore, which would you rather have?
Well, if it was something like the MP3.COM site, there wouldn't be another centralized source for replacements for the .mp3 files, so I don't see the issue. If the backup is corrupted, then you'd go to the next-oldest backup...
What do you mean, it refuses to pack? It gives an error or something?
No error, and no pack. I haven't had a lot of time to go into it ...
Anyone else out there have this problem??? Hope it isn't a serious bug...!
Also, importantly, version control!
you can version control the metadata.
That's another ball of worms - syncronizing with the external data (e.g. what if the file moves/renames/changes? Do you automatically scan? Or have a maintainance procedure of some kind to check? And have error handling, etc.?) I haven't looked at the code of ExtFile or LocalFS, don't know how they deal with that.
pack every day, and you may as well forget about an undo from yesterday.
Right, which is bad for all the other stuff you might want to undo...
I'm aiming at uploading of audio as something with a little more control (like a review process), so having 1000 versions of the same file unpacked isn't an issue, and the versioning is important.
Like I said, you can make it so the metadata is version-controllable. 1000 copies at 2MB=2GB ... for ONE lousy MP3? While _you_ may be fine with that, most people ar enot. Still, it is your choice. Multiply that by, say a hundred mp3s ...
Well, I was overstating the case. But I find it typical that, for example, one might mistakenly put up the wrong mix of a song, and have to pull it quickly... or that a mistake with some meta data, like publisher name or song title, is made once or twice before it's gotten right.
I wonder if, to get super-tweeky, it would be possible for a 'partial pack', where you could follow the 'thread' of a particular object or set of objects, and just pack those?
Except that you are then you are using quota on the _entire_ Data.fs. With an ExtFile you can have a seperate mountpoint that has a separate quota control. say I only want up to X Gb of mp3 files, but would also like up toX Gb of other objects, such as ExtImage objects. by limiting the data.fs, you can't do that very well. Additionally, maybe I don't want 1.5-2.5 Gb of a single song stored. ;^)~
A related thing I was thinking about was how to limit file size of individual files, and individual user quotas. My answer to that one was: 1. filesize would be limited by the 30 minute built-in timeout of zope... you can do the math... as long as you have enough storage for one of those (for each zope thread) you won't die... then you could, after upload, check the filesize of the file (either internal or external), and if it's too big, delete it and warn the user. 2. again, you can have methods to scan the user's area and add up his file storage, and perhaps disable uploading if it's over.
To me, it seems you could integrate these types of controls better with the site functionality if everything is inside zope.
Anyway, I was just kicking out some real-world reasons whyan ExtMP# type file would be (more) useful. Feel free to disagree, just understand that there are many of us out here here who have very valid reasons for not storing file objects that average over 1MB in the ZODb, especially when we are talking about potentially thousands, as you indicated. I've gone through all these issues in developing a Zope Advertising Product.
I'm looking at this issue keenly, and trying to dig down to the roots on these issues... I don't think there are easy answers, but overall I think the ZODB is underappreciated, and I'm trying to get a better grip on it's limitations. I'm still not clear on how the size of a python object is a factor (as opposed to the number of objects in data.fs), though it's pretty clear to me that anything that passes thru zope (e.g. via ExtFile) is at some point inside zope and therefore the same, at that point, as something stored in data.fs, as far as speed and performance goes.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- _________________________________________________ peter sabaini, mailto: sabaini@niil.at -------------------------------------------------