CoreSession + Non-undoable databases
I'm looking into CoreSessionTracking, and like what I see, but I've got a question. I want to have sessions persist beyond browser shut-off, which means I'll need a Session Data Container. In the docs, it's mentioned that the Data Container should be instantiated inside a non-undoing database. Well, I want to keep the undoability of the ZODB for most of my site, so how do I mount a non-undoable database somewhere in the tree? I've looked around at zope.org, and I've run into MountedStorage, but it seems to be out of date ("need a very recent CVS, the 2.2-pre releases aren't good enough," etc), so I'm wondering if there's a better way, or at least a Howto somewhere. Thanks again! -CJ WOW: Rapacious | A priest advised Voltaire on his death bed to apocalyptech.com/wow | renounce the devil. Replied Voltaire, "This pez@apocalyptech.com | is no time to make new enemies."
Ugh! This isn't as easy as it should be. First of all, you need to get a reasonably-initialized Data.fs made. You do this by installing a "scratch" copy of Zope using an alternate storage as the "main" storage (I recommend Packless Berkeley Storage that ships with the bsddb3Storage package (http://www.zope.org/Products/bsddb3Storage)). In other words, install a new "scratch" copy of Zope as if you were going to use an alternate storage to back the scratch Zope's main ZODB. Instructions for doing this exist in each storage package; and for different storages this involves setting up what's required for that storage -- eg for PacklessBerkeleyStorage, you need to install and set up Sleepycat's BerkeleyDB 3.X and the pybsddb module. Once you've got the required software installed, and the storage is working under the new "scratch" Zope instance (you can browse objects in the "new" ZODB), you've almost got the ZODB initialized properly. To finish, add a folder within the newly-initialized ZODB named "sessionData". Then you need to mount this newly-created ZODB within your *real* Zope instance (you can do away with the "scratch" Zope instance, just keep the files created by the alternate storage implementation). External Mount (http://www.zope.org/Members/hathawsh/ExternalMount) seems to be a reasonable way to do this. It's not documented very well, but the idea is that you create a mount by creating an exmount that returns a ZODB "DB" object. For example, to create a new database that returns a Packless-backed database, you'd: - install the ExMount product - add a file to your Extensions folder named "sessionDB" with the following contents (the actual semantics below might be wrong, it's just an example): import ZODB def getSessionDB(): return ZODB.DB.DB(Products.bsddb3Storage.Packless.Packless('session', '/path/to/packless/files')) - create an External Mount object in your ZODB (choose "Mount via External Method" from the Add list) with the options: Module: sessionDB Function: getSessionDB Path: /sessionData - Click add. You should now have a mounted database as "sessionData". - Add an external data container to the mounted database via the mgmt interface (e.g. at '/sessionData/sdc') - point your session data manager at the external data container by changing the path to whatever object you created in the last step. That's it. I've also put up a howto with this content at http://www.zope.org/Members/mcdonc/HowTos/UseExternalMountWithCST ----- Original Message ----- From: "CJ Kucera" <pez@apocalyptech.com> To: <zope@zope.org> Sent: Tuesday, October 09, 2001 1:01 PM Subject: [Zope] CoreSession + Non-undoable databases
I'm looking into CoreSessionTracking, and like what I see, but I've got a question.
I want to have sessions persist beyond browser shut-off, which means I'll need a Session Data Container. In the docs, it's mentioned that the Data Container should be instantiated inside a non-undoing database. Well, I want to keep the undoability of the ZODB for most of my site, so how do I mount a non-undoable database somewhere in the tree?
I've looked around at zope.org, and I've run into MountedStorage, but it seems to be out of date ("need a very recent CVS, the 2.2-pre releases aren't good enough," etc), so I'm wondering if there's a better way, or at least a Howto somewhere.
Thanks again! -CJ
WOW: Rapacious | A priest advised Voltaire on his death bed to apocalyptech.com/wow | renounce the devil. Replied Voltaire, "This pez@apocalyptech.com | is no time to make new enemies."
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Tue Oct 9 12:43:16 2001, Chris McDonough wrote:
Ugh! This isn't as easy as it should be.
Excellent; thanks for your help. It appears to be working. Some notes: 1) BerkeleyDB installs by default into /usr/local/BerkeleyDB.3.2 (obviously if you're installing a different version that'll be slightly different). This means that to compile bsddb3 you have to do the following: $ python setup.py build_ext --inplace --berkeley-db=/usr/local... What the bsdbb3 README files don't tell you is that you have to pass in the --berkeley-db= line to the "python setup.py install" command as well, or it won't be built properly. To find out if it's been built properly, cd into "bsddb3/{arch}/bsddb3" [1] and: $ nm _db.so | grep db_create If you get a line like "000242e0 T db_create" you're okay. If you get one like " U db_create" there's problems because it wasn't linked properly. The "_db.so" created inside the "build" directory during the initial "build_ext" step doesn't seem to actually have anything to do with anything; the one under "bsddb3" gets compiled at the "install" step. 2) The bsddb3Storage READMEs are a little obtuse. [2] When you've untarred the package, you'll have a "bsddb3Storage" directory with just .py files in it. THAT's the directory to copy into {ZOPE_HOME}/lib/python/Products. 3) Also, the "custom_zodb.sample" mentioned in the docs doesn't actually exist. It's "custom_zodb.py" in the current release (1.0beta4), and the syntax of the file doesn't work in Python < 2 (so if you're using Python 2, you should be okay). The custom_zodb.py I ended up using is here:
from bsddb3Storage.Packless import Packless #from bsddb3Storage.Minimal import Minimal as ConcreteStorage #from bsddb3Storage.Full import Full as ConcreteStorage
import os env = os.path.join('var', 'bsddb3Storage')
Storage = Packless(name='BerkeleyStorage', env=env)
Note that the "env" line is telling the BerkeleyDB that it's data directory will be in "var/bsddb3Storage" (your Data.fs is in "var"). 4) When you've created the "sessionData" folder in BerkeleyDB, shut down your temporary Zope instance and I recommend tarring up the bsddb3Storage directory under var, to move it over to your "real" zope instance. For instance: $ cd $TEMP_ZOPE/var; tar zcvf ../berkeleydb.tgz bsddb3Storage 5) Here's the correct syntax for "Extensions/sessionDB":
import ZODB from bsddb3Storage import Packless
def getSessionDB(): return ZODB.DB(Packless.Packless('session','berkeleyStorage'))
Note that "berkeleyStorage" is the name of the directory where the BerkeleyDB is going to be located (was "var/bsddb3Storage" on your "temp" Zope instance earlier). So, using my tarfile from before, here's basically what I did: $ cd $ZOPE_HOME; tar zxf ../temp_zope/berkeleydb.tgz $ mv bsddb3Storage berkeleyStorage So for me, I put the BerkeleyDB Storage directory in the same directory as var, lib, zpasswd.py, etc . . . 6) When it's been inserted into your tree, for all intents and purposes it looks EXACTLY like just a regular folder, as far as I can tell, which threw me off. There's even an "undo" tab and everything. So be careful; there doesn't seem to be an indication that it's actually an Externally Mounted folder as opposed to a regular one. In any case, after all that it appears to be working. You might want to update your HOWTO with what I've mentioned here. Thanks very much! -CJ [1] Replace "{arch}" with the directory there. On my system, it was "lib.linux-i686-1.5" [2] I lie. They're astonishingly useless. The READMEs distributed with 1.0beta4 are actually the 1.0beta3 files, which may account or most if not all of the problems. Under the "docs" directory there's a "PacklessReadme.txt" that you're much better off not reading. It makes references like "Once you've untarred the product, you should have a subdirectory named 'PacklessBerkeley3Storage'" which turn out to be patently false. So anyway . . . [3] [3] I'm being a bit harsh; they're not really THAT bad, just overly confusing at times. :) WOW: Rapacious | A priest advised Voltaire on his death bed to apocalyptech.com/wow | renounce the devil. Replied Voltaire, "This pez@apocalyptech.com | is no time to make new enemies."
Excellent, I'll add these notes to the HowTo. I really appreciate it. CJ Kucera wrote:
On Tue Oct 9 12:43:16 2001, Chris McDonough wrote:
Ugh! This isn't as easy as it should be.
Excellent; thanks for your help. It appears to be working. Some notes:
1) BerkeleyDB installs by default into /usr/local/BerkeleyDB.3.2 (obviously if you're installing a different version that'll be slightly different). This means that to compile bsddb3 you have to do the following:
$ python setup.py build_ext --inplace --berkeley-db=/usr/local...
What the bsdbb3 README files don't tell you is that you have to pass in the --berkeley-db= line to the "python setup.py install" command as well, or it won't be built properly. To find out if it's been built properly, cd into "bsddb3/{arch}/bsddb3" [1] and:
$ nm _db.so | grep db_create
If you get a line like "000242e0 T db_create" you're okay. If you get one like " U db_create" there's problems because it wasn't linked properly. The "_db.so" created inside the "build" directory during the initial "build_ext" step doesn't seem to actually have anything to do with anything; the one under "bsddb3" gets compiled at the "install" step.
2) The bsddb3Storage READMEs are a little obtuse. [2] When you've untarred the package, you'll have a "bsddb3Storage" directory with just .py files in it. THAT's the directory to copy into {ZOPE_HOME}/lib/python/Products.
3) Also, the "custom_zodb.sample" mentioned in the docs doesn't actually exist. It's "custom_zodb.py" in the current release (1.0beta4), and the syntax of the file doesn't work in Python < 2 (so if you're using Python 2, you should be okay). The custom_zodb.py I ended up using is here:
from bsddb3Storage.Packless import Packless #from bsddb3Storage.Minimal import Minimal as ConcreteStorage #from bsddb3Storage.Full import Full as ConcreteStorage
import os env = os.path.join('var', 'bsddb3Storage')
Storage = Packless(name='BerkeleyStorage', env=env)
Note that the "env" line is telling the BerkeleyDB that it's data directory will be in "var/bsddb3Storage" (your Data.fs is in "var").
4) When you've created the "sessionData" folder in BerkeleyDB, shut down your temporary Zope instance and I recommend tarring up the bsddb3Storage directory under var, to move it over to your "real" zope instance. For instance:
$ cd $TEMP_ZOPE/var; tar zcvf ../berkeleydb.tgz bsddb3Storage
5) Here's the correct syntax for "Extensions/sessionDB":
import ZODB from bsddb3Storage import Packless
def getSessionDB(): return ZODB.DB(Packless.Packless('session','berkeleyStorage'))
Note that "berkeleyStorage" is the name of the directory where the BerkeleyDB is going to be located (was "var/bsddb3Storage" on your "temp" Zope instance earlier). So, using my tarfile from before, here's basically what I did:
$ cd $ZOPE_HOME; tar zxf ../temp_zope/berkeleydb.tgz $ mv bsddb3Storage berkeleyStorage
So for me, I put the BerkeleyDB Storage directory in the same directory as var, lib, zpasswd.py, etc . . .
6) When it's been inserted into your tree, for all intents and purposes it looks EXACTLY like just a regular folder, as far as I can tell, which threw me off. There's even an "undo" tab and everything. So be careful; there doesn't seem to be an indication that it's actually an Externally Mounted folder as opposed to a regular one.
In any case, after all that it appears to be working. You might want to update your HOWTO with what I've mentioned here.
Thanks very much!
-CJ
[1] Replace "{arch}" with the directory there. On my system, it was "lib.linux-i686-1.5"
[2] I lie. They're astonishingly useless. The READMEs distributed with 1.0beta4 are actually the 1.0beta3 files, which may account or most if not all of the problems. Under the "docs" directory there's a "PacklessReadme.txt" that you're much better off not reading. It makes references like "Once you've untarred the product, you should have a subdirectory named 'PacklessBerkeley3Storage'" which turn out to be patently false. So anyway . . . [3]
[3] I'm being a bit harsh; they're not really THAT bad, just overly confusing at times. :)
WOW: Rapacious | A priest advised Voltaire on his death bed to apocalyptech.com/wow | renounce the devil. Replied Voltaire, "This pez@apocalyptech.com | is no time to make new enemies."
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
I said:
2) The bsddb3Storage READMEs are a little obtuse. [2] When you've untarred the package, you'll have a "bsddb3Storage" directory with just .py files in it. THAT's the directory to copy into {ZOPE_HOME}/lib/python/Products.
Whoops, this is a little misleading (should have proofread it). When you untar the package, the "bsddb3Storage" directory is INSIDE the main directory that gets untarred. ie:
$ tar ztf bsddb3Storage-1.0beta4.tar.gz bsddb3Storage-1.0beta4/ bsddb3Storage-1.0beta4/bsddb3Storage/ ... (etc)
So that's the one to copy over. -CJ WOW: Rapacious | A priest advised Voltaire on his death bed to apocalyptech.com/wow | renounce the devil. Replied Voltaire, "This pez@apocalyptech.com | is no time to make new enemies."
I said:
What the bsdbb3 README files don't tell you is that you have to pass in the --berkeley-db= line to the "python setup.py install" command as well, or it won't be built properly. To find out if it's been built properly, cd into "bsddb3/{arch}/bsddb3" [1] and:
$ nm _db.so | grep db_create
(etc...) I built up the CST/BerkeleyDB stuff on a different box last night (the one I had intended to put it on in the first place; was trying it out on my own box first), and didn't have the problems I mentioned when trying to get bsddb3 compiled, so it could very well be that there was something hokey with my first box that was causing me problems. So you might not have to expect these problems. Also, on this box, the "python setup.py install" DIDN'T accept the "--berkeley-db=" option, so either distutils was set up much differently on the two boxes, or my memory was just a bit off when I wrote the original email. Therefore: You might want to put this in as a disclaimer on the Howto page . . . Once again, thanks! -CJ WOW: Rapacious | A priest advised Voltaire on his death bed to apocalyptech.com/wow | renounce the devil. Replied Voltaire, "This pez@apocalyptech.com | is no time to make new enemies."
Chris McDonough wrote:
the "main" storage (I recommend Packless Berkeley Storage that ships with the bsddb3Storage package (http://www.zope.org/Products/bsddb3Storage)).
I thought Packless was being phased out in favour of Minimal? Chris
I thought Packless was being phased out in favour of Minimal?
Maybe. Packless works now without needing a pack. Minimal has a ways to go before that time. When Minimal is supported and also "packless", then I'll recommend that instead. - C
participants (4)
-
Chris McDonough -
Chris McDonough -
Chris Withers -
CJ Kucera