On Fri, Jan 31, 2003 at 04:00:39PM +0900, Wankyu Choi wrote:
On Wed, Jan 29, 2003 at 03:32:20PM +0900, Wankyu Choi wrote:
I've been thinking:
1. Take out as much binary data as possible via External File ( my version, not the ones found in Zope products page ).
i can understand doing this if you're running FileStorage and want to avoid a single bloated Data.fs file, but since you're leaning towards DirectoryStorage, I think there's no reason to use ExternalFile at all - it just complicates things.
Hm.. your reply pretty much convinces me^^
Well, i wouldn't go that far. :) I should add the disclaimer that i have not actually used DirectoryStorage in production yet. I made the above statement based on the DS docs. I have only started evaluating options myself after our recent troubles with FS corruption. All this is IM-very-humble O, and YMMV and all that.
Now I'll be trying DirectoryStorage in earnest.
2. Store textual data ( raw data + its rendered version, etc ) in MySQL.
ditto
Why shouldn't I let DirectoryStorage also take care of this?
sorry, by "ditto" I meant to convey "same answer I gave to the last point". If DS performs reliably as advertised, it would completely obviate the need for storing anything outside the ZODB, from a scalability & maintenance standpoint at least. **HOWEVER** you are concerned about performance, and DS is even slower than FileStorage for writes. From the DS FAQ: """Intermittant writes are a factor of 1.5 slower. ... Under high write pressure the journal queue becomes a bottleneck, and performance degrades to 3 times slower than FileStorage. """ The question then becomes, what is "high write pressure"? And what does 3x slower than FS feel like to the user?
3. User data goes into MySQL.
ditto
Again why? In fact, I want to keep the existing user data where they are but am just curious what I would be missing if I didn't^^
again, by "ditto" I meant to convey "same answer I gave to the last point" :)
Okay, but is there anything I should be aware of before deciding on the directory storage as my storage of choice? The data will be both read- and write-intensive.
given what the DS FAQ says about write performance, I'd look into setting up a test server and bombard it with automated writes to see if it will handle the load you anticipate. But of course you were going to do that anyway. ;)
If most Zope users ( including yourself ) feel Filestorage is not a solution for more than 20GB of data,
hell, I don't like it for 2 GB of data. (see recent thread "POSKeyError II: Dead by Dawn" ... I wasn't kidding when I titled it that!)
Yeah, that POSKeyError thing scares the hell out of me ;-)
I could almost feel the server screaming "JOIIINNNN USSSSSS" in an eery DSP-treated voice. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's SNAZZY DEVIANT GIRL! (random hero from isometric.spaceninja.com)