Question about import code
Hi folks, I have hacked on the export code to allow you to export more than one object at a time. Now I'm trying to grok the import code. Can someone help me figure out what's going on? I have an export file in XML format that looks like this: <?xml version="1.0"?> <ZopeData> <record id="5602" aka="AAAAAAAAFeI="> [ first record's data here... ] </record> <record id="5603" aka="AAAAAAAAFeM="> [ second record's data here... ] </record> </ZopeData> If you export a folderish object, you'll see basically the same export format. (When you export a folder, you get all of the objects rooted in that folder.) The difference is that these two records are not related -- they are simply both contained in the same folder. The problem that happens on import is a KeyError on line 194 of this bit of code: lines 192-197 of lib/python/ZODB/ExportImport.py ooid=h[:8] if oids: oid=oids[ooid] if type(oid) is TupleType: oid=oid[0] else: oids[ooid]=return_oid=oid=new_oid() If I change this code to ooid=h[:8] if oids and oids.has_key(ooid): oid=oids[ooid] if type(oid) is TupleType: oid=oid[0] else: oids[ooid]=return_oid=oid=new_oid() then the import appears to succeed, but only the final object in the file is actually imported. I have to say that some comments would be nice in lib/python/ZODB/ExportImport.py Also, are both of these files used? lib/python/ExportImportXML.py lib/python/OFS/XMLExportImport.py Plus, I am having a hard time finding the right documentation for working directly with the ZODB to see what's going on. Can someone help point me in the right direction? Here's what I'm trying to accomplish: 1. Allow multiple objects to be exported at once (this is working) 2. Allow multiple objects to be imported at once (this is not) 3. Allow imports to overwrite existing objects (haven't tried this yet) Thanks in advance for any assistance. Fred -- Fred Wilson Horch mailto:fhorch@ecoaccess.org Executive Director, EcoAccess http://ecoaccess.org/ P.O. Box 2823, Durham, NC 27715-2823 phone: 919.419-8567
Fred Wilson Horch wrote:
The problem that happens on import is a KeyError on line 194 of this bit of code:
lines 192-197 of lib/python/ZODB/ExportImport.py
ooid=h[:8] if oids: oid=oids[ooid] if type(oid) is TupleType: oid=oid[0] else: oids[ooid]=return_oid=oid=new_oid()
If I change this code to
ooid=h[:8] if oids and oids.has_key(ooid): oid=oids[ooid] if type(oid) is TupleType: oid=oid[0] else: oids[ooid]=return_oid=oid=new_oid()
then the import appears to succeed, but only the final object in the file is actually imported.
Importing multiple objects simultaneously could result in a fair amount of confusion... But if you really think it's a good idea, if you used a list called return_oids instead, and appended to this list in the code above, you'd probably get what you're looking for. ooid=h[:8] if oids and oids.has_key(ooid): oid=oids[ooid] if type(oid) is TupleType: oid=oid[0] else: oids[ooid] = oid = new_oid() return_oids.append(oid) Then at the end of the function you'll want to dereference each of the OIDs and return a list of objects. Everything that uses import will have to be modified, unless you change the name of the method and provide a wrapper method with the existing signature. Note that for Zope 2.4 this code has changed a bit. Import is now done as a subtransaction rather than a transaction "on the side". Shane
Hi Shane, You wrote:
Importing multiple objects simultaneously could result in a fair amount of confusion...
Can you elaborate? When you import a folder, for example, it imports the folder and all objects in it. Why is it any different to import two unrelated objects? Would it be better to do this in two separate transactions? Or two subtransactions?
But if you really think it's a good idea, if you used a list called return_oids instead, and appended to this list in the code above, you'd probably get what you're looking for.
I tried this approach, too, and I get a ConflictError. (I can send the code and traceback if you're interested.)
Note that for Zope 2.4 this code has changed a bit. Import is now done as a subtransaction rather than a transaction "on the side".
Thanks for the heads up. When you say "subtransaction", a subtransaction of what? I'm familiar with subtransactions being atomic parts of a larger transaction. I'm not sure what this larger transaction would be in this case. If I start an import, does that start a transaction with each object's import being a subtransaction of the whole thing? (If that's the case, then I don't understand why importing several objects at once would be confusing.) Is there any documentation on how to export from and import to the ZODB? The code I'm wading through has a lot of extraneous stuff thrown in. I'd like to be able to just try exporting and importing objects from a simple Python script so I understand what's going on. Thanks, Fred -- Fred Wilson Horch mailto:fhorch@ecoaccess.org Executive Director, EcoAccess http://ecoaccess.org/ P.O. Box 2823, Durham, NC 27715-2823 phone: 919.419-8567
Fred Wilson Horch wrote:
You wrote:
Importing multiple objects simultaneously could result in a fair amount of confusion...
Can you elaborate? When you import a folder, for example, it imports the folder and all objects in it.
If there are some objects in the folder that have conflicting names and others that don't, what do you do? Also, if you're thinking about providing this functionality, what about spanning multiple folders? What if some folders already exist? The capability could change user expectations, you see, and you may need to deal with them. But that's just advice and you can freely ignore it. :-)
Thanks for the heads up. When you say "subtransaction", a subtransaction of what? I'm familiar with subtransactions being atomic parts of a larger transaction. I'm not sure what this larger transaction would be in this case. If I start an import, does that start a transaction with each object's import being a subtransaction of the whole thing?
That's right.
Is there any documentation on how to export from and import to the ZODB?
Perhaps. Look at the interfaces Wiki.
The code I'm wading through has a lot of extraneous stuff thrown in. I'd like to be able to just try exporting and importing objects from a simple Python script so I understand what's going on.
An external method could do it. Look at what _getCopy() does in OFS/CopySupport.py. Shane
participants (2)
-
Fred Wilson Horch -
Shane Hathaway