Re: [Zope] Zope's scalability across multiple web servers
Jason Spisak wrote:
Beyond the fact that this may not be a good idea, you might want to store the postings in Zope, but the moderation score on the FS, as it is likely to get overwritten A LOT (especially by moderation bots). You run the slight risk of a given moderation being overwritten
Please forgive my ignorance, but could you explain the essencial difference between writing to the ZODB and a Filesystem Backend. I'm just not getting why...
Basically, the ZODB is optimized in design for many reads and few writes, and the filesystem is not. This means that in many-write situations, the ZODB is working much harder than usual, and performance of the site will go down faster than under an equivalent number of reads.
statistically it should even out with as many 'up' moderations being discarded accidentally as 'down' ones.
Is it because there is no locking?
Yes.
And you prevent Zope from thrashing with many tiny committed transactions mucking up the 'Undo' tab.
I definately understand that part. I really appreciate this BTW. It's an area of the Zope scope (;-) that I have thought about for a while(storage).
I beleive that this tradeoff (transactional integrity vs. efficient writes) would be made irrelevant if the FS in question was either ReiserFS or the IRIX Journalling FS, but I don't really know enough about either to be certain. HTH, Michael Bernstein, Information Architect.
Michael Bernstein:
Basically, the ZODB is optimized in design for many reads and few writes, and the filesystem is not. This means that in many-write situations, the ZODB is working much harder than usual, and performance of the site will go down faster than under an equivalent number of reads.
That makes sense.
statistically it should even out with as many 'up' moderations being discarded accidentally as 'down' ones.
Is it because there is no locking?
Yes.
Okay. I get it. So if the files system took care of the journaling/transaction end of things then that would close that hole?
I beleive that this tradeoff (transactional integrity vs. efficient writes) would be made irrelevant if the FS in question was either ReiserFS or the IRIX Journalling FS, but I don't really know enough about either to be certain.
Is there currently a Product that uses the filesystem like you are talking about? FSSession? Is there a product to store the all my ZClass instances let's say on the file system right now? All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
On Mon, 1 May 2000, Jason Spisak wrote:
Is there currently a Product that uses the filesystem like you are talking about? FSSession? Is there a product to store the all my ZClass instances let's say on the file system right now?
FSSession does participate in Zope transaction machinery and can store arbitrary python objects. Modifications to an FSSession instance are only saved during the commit action. However I still have to do some tests to verify correct behaviour under errors raised during commit operations in other objects. Although I have not tried it I do not forsee any problems with storing Zclass instances in a session but then it won't be easy to manage them. Pavlos
Pavlos Christoforou writes:
On Mon, 1 May 2000, Jason Spisak wrote:
Is there currently a Product that uses the filesystem like you are talking about? FSSession? Is there a product to store the all my ZClass instances let's say on the file system right now?
FSSession does participate in Zope transaction machinery and can store arbitrary python objects.
That's goo to hear.
Modifications to an FSSession instance are only saved during the commit action. However I still have to do some tests to verify correct behaviour under errors raised during commit operations in other objects.
Although I have not tried it I do not forsee any problems with storing Zclass instances in a session but then it won't be easy to manage them.
So there isn't a product out there that allows what Michel B. is talking about? Where the objects are stored as files on the filesystem and the filesystem tools are used to manage quotas and the like. All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Jason Spisak wrote:
Michael Bernstein:
statistically it should even out with as many 'up' moderations being discarded accidentally as 'down' ones.
Is it because there is no locking?
Yes.
Okay. I get it. So if the files system took care of the journaling/transaction end of things then that would close that hole?
AFAIK. In theory.
I beleive that this tradeoff (transactional integrity vs. efficient writes) would be made irrelevant if the FS in question was either ReiserFS or the IRIX Journalling FS, but I don't really know enough about either to be certain.
Is there currently a Product that uses the filesystem like you are talking about? FSSession?
Yes, FSSession stores session info on the FS.
Is there a product to store the all my ZClass instances let's say on the file system right now?
Not that I'm aware of, however there is a product that will store and access certain file-like objects from the FS: LocalFS. In fact, it has a very cool feature that instantiates .xml files as XMLDocuments when they're retrieved from the FS, so you can do things like display a tree of the nodes within an XML file inside the Zope management interface. HTH, Michael Bernstein.
Michael:
Okay. I get it. So if the files system took care of the journaling/transaction end of things then that would close that hole?
AFAIK. In theory.
I beleive that this tradeoff (transactional integrity vs. efficient writes) would be made irrelevant if the FS in question was either ReiserFS or the IRIX Journalling FS, but I don't really know enough about either to be certain.
Is there currently a Product that uses the filesystem like you are talking about? FSSession?
Yes, FSSession stores session info on the FS.
I'll have a look. I don't know jack about low level filesystem stuff, but maybe a cross between FSSession and LocalFS with an XFS/ReiserFS is what I'm thinking about.
Is there a product to store the all my ZClass instances let's say on the file system right now?
Not that I'm aware of, however there is a product that will store and access certain file-like objects from the FS: LocalFS. In fact, it has a very cool feature that instantiates .xml files as XMLDocuments when they're retrieved from the FS, so you can do things like display a tree of the nodes within an XML file inside the Zope management interface.
That sounds fantastic. Thanks again, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
On Mon, May 01, 2000 at 03:44:11PM +0000, Jason Spisak wrote:
Is there currently a Product that uses the filesystem like you are talking about? FSSession? Is there a product to store the all my ZClass instances let's say on the file system right now?
If there were one, it wouldn't be a Product. What you're looking for is a Storage for ZODB - look in your zope-home under /lib/python/ZODB. You can just write storage methods and plug them in. OTOH you can't (yet) store parts of your Zope tree in one storage and parts in other; you would have to move the whole thing from Data.fs to whatever you're going to use. Still I'd like to see a "filesystem directory tree storage" (but what about version control? Undo? The whole transaction infrastructure?) Personally, I think the best sollution for us is a storage based on XDelta and XDFS (check out modules xdelta, prcs2 and xdfs from the Gnome CVS and check it out). And there is a draft paper by the author describing it... uh... where is it? Oh, here: http://www.cs.berkeley.edu/~jmacd/masters.ps.gz I've been dreaming of a XDFS storage for the past few weeks :-) I'm even considering offering to hire Josh MacDonald (the author of XDFS) to write it, I don't know whether he'd be available but... []s, |alo +---- -- Hack and Roll ( http://www.hackandroll.org ) News for, uh, whatever it is that we are. http://www.webcom.com/lalo mailto:lalo@hackandroll.org pgp key in the personal page Brazil of Darkness (RPG) --- http://zope.gf.com.br/BroDar
On Tue, 2 May 2000, Lalo Martins wrote:
Still I'd like to see a "filesystem directory tree storage" (but what about version control? Undo? The whole transaction infrastructure?)
That would be the difficult part. I suppose one way is to use the 'spare' attributes on some new filesystems to store transaction ids. Then commands like rm/mv/cp etc will be replaced with their transactional versions so they will ensure transactional integrity. In other words I have no clue ... Regarding archiving and distributing Data.fs files very efficient algorithms specific to Data.fs can be written. As Data.fs only appends data (with the exception of packing) one can quickly locate any new appends (based on transaction ids) and distribute them to different servers. Additionaly the changes can be stored incrementaly so you could recreate a Data.fs file for a specified date. I really wished I had some time to work on it ... I don't thing it would be very diffult. In other words I am clueless again ... Pavlos
Pavlos:
On Tue, 2 May 2000, Lalo Martins wrote:
Still I'd like to see a "filesystem directory tree storage" (but what about version control? Undo? The whole transaction infrastructure?)
That would be the difficult part. I suppose one way is to use the 'spare' attributes on some new filesystems to store transaction ids. Then commands like rm/mv/cp etc will be replaced with their transactional versions so they will ensure transactional integrity. In other words I have no clue ...
Me neither in that respect.
Regarding archiving and distributing Data.fs files very efficient algorithms specific to Data.fs can be written.
This is what I'm talking about. Splitting the Data.fs into pieces and maybe even each object has it's own piece with revisions of that object stored in it's piece. This offers some interesting pluses. Anyone see anything really glaringly wrong with this?
As Data.fs only appends data (with the exception of packing) one can quickly locate any new appends (based on transaction ids) and distribute them to different servers. Additionaly the changes can be stored incrementaly so you could recreate a Data.fs file for a specified date.
Oooh.
I really wished I had some time to work on it ... I don't thing it would be very diffult. In other words I am clueless again ...
I am just finishing the "learning Python" O'Reilly book (just to let you know where my skills are) But this sounds too interesting not to try. Again, does anyone see anything glaringly wrong with this? All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
On Tue, 2 May 2000, Jason Spisak wrote:
This is what I'm talking about. Splitting the Data.fs into pieces and maybe even each object has it's own piece with revisions of that object stored in it's piece. This offers some interesting pluses. Anyone see anything really glaringly wrong with this?
I would suggest that splitting should be done at the transaction level not the individual object level. Initially one might keep the whole system as is, and run a seperate process that monitors data.fs and caches a map of transaction ids to byte offsets. A client can place a request to that process passing as parameters a transaction id (and a few other things) and the server process returning the data segment of Data.fs that occured after that transaction. If packing has occured the server can notify the client about it, and then it is up to the client to deal with 'its own' Data.fs. The client upon receiving the data can either append them to the Data.fs or store them individually. A script then can use those segments to reconstruct a snapshot Data.fs for a specified time. What you are suggesting is more interesting but I think also more difficult. Pavlos
Pavlos:
I would suggest that splitting should be done at the transaction level not the individual object level.
Is that because this other stuff would mean a comlpete re-write of Zope interface to the ZODB?
Initially one might keep the whole system as is,
You mean a single ZODB file?
and run a seperate process that monitors data.fs and caches a map of transaction ids to byte offsets. A client can place a request to that process passing as parameters a transaction id (and a few other things) and the server process returning the data segment of Data.fs that occured after that transaction. If packing has occured the server can notify the client about it, and then it is up to the client to deal with 'its own' Data.fs. The client upon receiving the data can either append them to the Data.fs or store them individually. A script then can use those segments to reconstruct a snapshot Data.fs for a specified time.
Very interesting proposal.
What you are suggesting is more interesting but I think also more difficult.
Isn't that always the way? Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Regarding archiving and distributing Data.fs files very efficient algorithms specific to Data.fs can be written.
This is what I'm talking about. Splitting the Data.fs into pieces and maybe even each object has it's own piece with revisions of that object stored in it's piece. This offers some interesting pluses. Anyone see anything really glaringly wrong with this?
I've been sorta following this conversation for a bit now.. Now I'm not a python/zope expert, nor a filesystem expert, and I haven't even looked at the code.. but I do see a couple things that could be wrong with this approach.. 1. Number of file handles.. More resources required.. What happens when several 1000 objects are needed to be access, and you have to close some of the ones you have open? More complexity. 2. amount of time determining which file to open/access.. With only having a single file, you don't have to make any desicions. Speed by simplicity. Do the other advantages of being FS based outwiegh this? I don't know.. --John
On Tue, 2 May 2000, John Sutherland wrote:
2. amount of time determining which file to open/access.. With only having a single file, you don't have to make any desicions. Speed by simplicity.
In a single file one still needs a mechanism to determine location and access an object. In a filesytem based storage the filesytem will be responsible of finding the file given its 'id'.
Do the other advantages of being FS based outwiegh this? I don't know..
I am not sure. Maybe not. Even if there was such an option I don't see how some useful FS based tools like CVS will play nicely with the transaction machinery of ZODB. However I am pretty confident that storing parts like session info, frequently changing property sheets etc, in the current FileStorage implementation is not very efficient. pavlos
John:
I've been sorta following this conversation for a bit now.. Now I'm not a python/zope expert, nor a filesystem expert, and I haven't even looked at the code.. but I do see a couple things that could be wrong with this approach..
1. Number of file handles..
As far as I know, this is an OS limitation. 64bit os and filesystem like XFS has more than enough handles.
More resources required..
What exactly do you mean by more resources? I am just talking about the exact same data for an object stored in the current configuration, only split into multiple files.
What happens when several 1000 objects are needed to be access, and you have to close some of the ones you have open? More complexity. 2. amount of time determining which file to open/access.. With only having a single file, you don't have to make any desicions. Speed by simplicity.
True, but object caching happens now with the ZODB. How does Zope know "where in the Data.fs file" to find a specific object? It can't be any more of a laborious process in a 2 GB ZODB than finding where on the filesystem a particular file is. All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Jason Spisak wrote:
Pavlos:
On Tue, 2 May 2000, Lalo Martins wrote:
Regarding archiving and distributing Data.fs files very efficient algorithms specific to Data.fs can be written.
This is what I'm talking about. Splitting the Data.fs into pieces and maybe even each object has it's own piece with revisions of that object stored in it's piece. This offers some interesting pluses. Anyone see anything really glaringly wrong with this?
Well, what about 'move' transactions (cut-and-paste)? Michael Bernstein.
Michael:
Jason Spisak wrote:
Pavlos:
On Tue, 2 May 2000, Lalo Martins wrote:
Regarding archiving and distributing Data.fs files very efficient algorithms specific to Data.fs can be written.
This is what I'm talking about. Splitting the Data.fs into pieces and maybe even each object has it's own piece with revisions of that object stored in it's piece. This offers some interesting pluses. Anyone see anything really glaringly wrong with this?
Well, what about 'move' transactions (cut-and-paste)?
How does the ZODB handle them now? Are the locations of objects dependent on their location in the ZODB file? If not, then there is no difference. I'm not sure how relationship data (container, etc...) is stored in the ZODB. Is it stored with the object? Anyone? All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Lalo:
If there were one, it wouldn't be a Product. What you're looking for is a Storage for ZODB - look in your zope-home under /lib/python/ZODB. You can just write storage methods and plug them in.
Thanks for the heads up.
OTOH you can't (yet) store parts of your Zope tree in one storage and parts in other; you would have to move the whole thing from Data.fs to whatever you're going to use.
Isn't that what SheetProviders and RIPP is for? Ty?
Still I'd like to see a "filesystem directory tree storage" (but what about version control? Undo?
I just want to see filesystems storage of properties and things that are high-write. The current undo interface isn't feasable in this environment anyway. The transaction I want to changes is 100 pages back by the time I want to change it.
The whole transaction infrastructure?)
Isn't that kind of like a journaling filesystem?
Personally, I think the best sollution for us is a storage based on XDelta and XDFS (check out modules xdelta, prcs2 and xdfs from the Gnome CVS and check it out). And there is a draft paper by the author describing it... uh... where is it? Oh, here: http://www.cs.berkeley.edu/~jmacd/masters.ps.gz
That's new to me. Very interesting. Let me check it out.
I've been dreaming of a XDFS storage for the past few weeks :-) I'm even considering offering to hire Josh MacDonald (the author of XDFS) to write it, I don't know whether he'd be available but...
I heard about some guy named linus working for a chip company. :-) All my best, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
participants (5)
-
Jason Spisak -
John Sutherland -
Lalo Martins -
Michael Bernstein -
Pavlos Christoforou