Re: [Zope] Understand ZODB Query?
you're right, i'm still thinking tabularly. I'm basically looking for ways how to do database query (select, update, create, etc). Of course i'm still thinkin in 'relation database' way as I'm just starting with OODB.
From my latest reading, the ZODB works like a 'dictionary': -In this way, I can save any objects into this dictionary. -the key for the dictionary is the 'unique key' while the value is the object. -In order to perform any query, I need to parse through the dictionary
My questions: -wouldn't it be too slow? -How if I don't need any key? -How if I have multiple keys? This doesn't seem so easy. Probably I need to read ZCatalog and get some understanding from its query. thanks. Heri
- I can easily perform SQL statement to lookup for some information such as, total employee, who are above 30 years old, and who has the Lastname of 'Smith'. - I can easily retrieve information from multiple tables and combine it as data objects.
But how can I do this in ZODB?
1) If your data set is not too big, you can do it programmatically Python: set = folder.objectValues() results = [] for elt in set: if (elt.age > 30) and (elt.lastname=='Smith'): results.append(elt) return elt One can easily replace '30' and 'Smith' with parameters to get a more useful script. 2) Use ZCatalog for this if you have larger sets to avoid unnecessary traversals. But I suspect the larger problem is that you're thinking tabularly. Trying to make a 1-to-1 translation between an object-storage scheme and a relational-storage scheme isn't always (or maybe usually) going to work. I know no magic processes for this, unfortunately. --jcc
I have been getting a POSKeyError when trying to list my Members directory through the ZMI. I backed up everything packed the ZODB and turned on debugging. Looks like there is a bad object. When I run fstest.py: 119583146L object serialno 0x034aab9e4b7e624c does not matchtransaction id 0x034ac5e367948900 When I run it in debug mode I see it is this object: \x00\x00\x00\x00\x00\x01\xef\xce I grep'd on both of those things in my Data.fs but cannot find it. So, how do I (or should I) delete that object? BZ
have you tried the debugger? you can find out more about the object that way. suppose you are in the debugger and the zope root is bound to app:: app._p_jar['\x00\x00\x00\x00\x00\x01\xef\xce'] that grabs the object "directly" and you might be able to find out more that way. jens On Wednesday, Mar 12, 2003, at 07:01 US/Eastern, BZ wrote:
I have been getting a POSKeyError when trying to list my Members directory through the ZMI. I backed up everything packed the ZODB and turned on debugging. Looks like there is a bad object.
When I run fstest.py: 119583146L object serialno 0x034aab9e4b7e624c does not matchtransaction id 0x034ac5e367948900
When I run it in debug mode I see it is this object: \x00\x00\x00\x00\x00\x01\xef\xce
I grep'd on both of those things in my Data.fs but cannot find it. So, how do I (or should I) delete that object?
BZ
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Ok... a clue. I went into the debugger and all I saw was my current session opened. Look at the cache detail I found the object: 1 '\x00\x00\x00\x00\x00\x01\xef\xce' 1 G Products.CMFCore.PortalFolder.PortalFolder That is the object I am having problems with, but I have other Portal Folders which are fine. Another clue would be great... BZ
have you tried the debugger? you can find out more about the object that way. suppose you are in the debugger and the zope root is bound to app::
app._p_jar['\x00\x00\x00\x00\x00\x01\xef\xce']
that grabs the object "directly" and you might be able to find out more that way.
jens
On Wednesday, Mar 12, 2003, at 07:01 US/Eastern, BZ wrote:
I have been getting a POSKeyError when trying to list my Members directory through the ZMI. I backed up everything packed the ZODB and turned on debugging. Looks like there is a bad object.
When I run fstest.py: 119583146L object serialno 0x034aab9e4b7e624c does not matchtransaction id 0x034ac5e367948900
When I run it in debug mode I see it is this object: \x00\x00\x00\x00\x00\x01\xef\xce
I grep'd on both of those things in my Data.fs but cannot find it. So, how do I (or should I) delete that object?
BZ
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
i don't know what you mean by "all i saw was my current session opened". have you tried something like... ob = app._p_jar['\x00\x00\x00\x00\x00\x01\xef\xce'] ob.getPhysicalPath() etc... ? jens On Wednesday, Mar 12, 2003, at 08:40 US/Eastern, BZ wrote:
Ok... a clue.
I went into the debugger and all I saw was my current session opened. Look at the cache detail I found the object:
1 '\x00\x00\x00\x00\x00\x01\xef\xce' 1 G Products.CMFCore.PortalFolder.PortalFolder
That is the object I am having problems with, but I have other Portal Folders which are fine.
Another clue would be great...
BZ
have you tried the debugger? you can find out more about the object that way. suppose you are in the debugger and the zope root is bound to app::
app._p_jar['\x00\x00\x00\x00\x00\x01\xef\xce']
that grabs the object "directly" and you might be able to find out more that way.
jens
On Wednesday 12 March 2003 12:01 pm, BZ wrote:
I have been getting a POSKeyError when trying to list my Members directory through the ZMI. I backed up everything packed the ZODB and turned on debugging. Looks like there is a bad object.
When I run fstest.py: 119583146L object serialno 0x034aab9e4b7e624c does not matchtransaction id 0x034ac5e367948900
These are hex encoded timestamps. It indicates that part of a transaction from Tue Feb 4 20:46:17 2003 is mixed up with the transaction record from Sun Feb 9 12:51:24 2003 Do these times seem significant?
So, how do I (or should I) delete that object?
This is indicating a bad FileStorage, not a bad object. Deleting an object might not make the problem go away, and might make it worse. A few people have been reporting this recently, and sadly noone has a good answer for this FileStorage problem. If you can reproduce the problem starting with a clean FileStorage then I am sure a FileStorage developer will take a look. -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson
Bad filestorage eh? I was going to use fsrecover.py (need to download, not included with Zope 2.5.1) but wanted to see if there was anything else first. I read the other thread about POSKeyErrors and didn't find a solution. BZ
On Wednesday 12 March 2003 12:01 pm, BZ wrote:
I have been getting a POSKeyError when trying to list my Members directory through the ZMI. I backed up everything packed the ZODB and turned on debugging. Looks like there is a bad object.
When I run fstest.py: 119583146L object serialno 0x034aab9e4b7e624c does not matchtransaction id 0x034ac5e367948900
These are hex encoded timestamps. It indicates that part of a transaction from Tue Feb 4 20:46:17 2003 is mixed up with the transaction record from Sun Feb 9 12:51:24 2003
Do these times seem significant?
So, how do I (or should I) delete that object?
This is indicating a bad FileStorage, not a bad object. Deleting an object might not make the problem go away, and might make it worse.
A few people have been reporting this recently, and sadly noone has a good answer for this FileStorage problem. If you can reproduce the problem starting with a clean FileStorage then I am sure a FileStorage developer will take a look.
-- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
you're right, i'm still thinking tabularly.
It can be a hurdle. In a relational system, your data is pretty much global, and you relate it by keys and the other tricks of the RDB. In an object system like Zope, you (can) store information hierarchically. It's containment (sometimes with reference) and context that do the work here. (There are middle grounds like EJBs, where there are objects, but they're all really in the same pile, and have no real containment.)
I'm basically looking for ways how to do database query (select, update, create, etc). Of course i'm still thinkin in 'relation database' way as I'm just starting with OODB.
There are analogues: "select" basically means "find these records". In an object system, you can either contain things so that this is intrinsic or nearly so (by design) and do some simple filtering on the results or do some sort of search (see below). "update" is just a method called on an object you already have a handle on. "create" is a method called on an object that can contain other objects. You must have a handle on it, too.
From my latest reading, the ZODB works like a 'dictionary': -In this way, I can save any objects into this dictionary. -the key for the dictionary is the 'unique key' while the value is the object. -In order to perform any query, I need to parse through the dictionary
If you're working low-level at the ZODB, that's right. But since you're probably getting at it through Zope, it's better to think of it as an object tree (almost like a filesystem, but don't get too wrapped up in that metaphor.) Even forget the fact that there's storage behind it and pretend like you're looking at the guts of a running program (which you are!) To find things in this environment, you can walk the tree yourself, or get someone else (like ZCatalog) to do it for you.
My questions: -wouldn't it be too slow?
Not if you do it right. If all your relevant records are contained in some object, then you merely need ask it for its contents. ZCatalog is an index based system, which is pretty darn fast since all the hard work is done ahead of time.
-How if I don't need any key? -How if I have multiple keys?
The whole key question is kind of orthogonal to Zope. All objects really have one and only one unique key: their path. Everything else is application-level.
This doesn't seem so easy.
It's probably too easy. You just aren't used to thinking about it this way. I'll do a little summary since the above is so scattered: Zope is really a live object system which happens to have a persistent backing. But forget that, and think like you're in a program and not a database. You can add your data by 1) creating a Product whose instances store a single record, and adding a heap of these in one or more folders, perhaps hierarchically. These can be seen and edited via the ZMI and are catalogable (automatically if you do it right.) This is what the default Products do. You might have a Person product, and folders for each Office, with instances of Persons in them. 2) creating a Product with attributes that store your data and retrieve it. Say a products People which has a dictionary of all the people you add to it, and methods for adding, finding, and changing people. You may have to tell the persistence machinery when things have changed. This data can't be cataloged, and you have to take care of the machinery yourself. If you implement by dictionary, this can be very relational; if you implement by objects, not so much. 3) Using existing products to do this for you, by either method above. You can relate data by 1) parent-child relationships 2) containment of attributes, possibly by unique key 3) holding references to objects, paths to objects, or by using the Relationship product. There. That's a little less scattered. Should be helpful, but please don't pretend it's authoritative. --jcc
participants (5)
-
BZ -
Heri -
J Cameron Cooper -
Jens Vagelpohl -
Toby Dickenson