Storing lots of big objects in containers
Hi guys, A product I'm currently developing has a rather big (in many ways) problem. A product instance may store many, (thousands) of big objects, ranging from 1-50 MB in size. Now, whenever the product instance is accessed, with say for example 100 objects with a size of 2MB each, Zope slows down to a crawl, and spends a lot of time loading (I assume) the objects from the ZODB, even though only 10 objects are actually accessed. ..I can't really use any external storages, as the reason for bringing these objects into the ZODB was to structure them and make then indexable by ZCatalog. And there's probably no time for ZPatterns. Someone suggested using BTreeFolder, but the documentation and examples for it are rather scarce. Does anyone know of products that make a good example use of BTreeFolder? Would BTreeFolder work for this? Are there better alternatives? Thank you for your time, Morten
On 24 Feb 2001, Morten W. Petersen wrote:
Someone suggested using BTreeFolder, but the documentation and examples for it are rather scarce. Does anyone know of products that make a good example use of BTreeFolder?
Can't you just subclass the BTree Folder as you would with OFS.Folder? I think you might be confusing the Zope BTree implementation with the BTree Folder Product?
[Erik Enge] | Can't you just subclass the BTree Folder as you would with OFS.Folder? | | I think you might be confusing the Zope BTree implementation with the | BTree Folder Product? I've tried subclassing BTreeFolder, but then, whenever the object is accessed, zope falls flat on its face. :-\ Tried again, now this error is raised: """ Zope Error Zope has encountered an error while publishing this resource. TypeError Sorry, a Zope error occurred. Traceback (innermost last): File /home/morten/Zope-2.3.0-src/lib/python/ZPublisher/Publish.py, line 222, in publish_module File /home/morten/Zope-2.3.0-src/lib/python/ZPublisher/Publish.py, line 187, in publish File /home/morten/Zope-2.3.0-src/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook (Object: CatalogAware) File /home/morten/Zope-2.3.0-src/lib/python/ZPublisher/Publish.py, line 171, in publish File /home/morten/Zope-2.3.0-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: retrieve_messages) File /home/morten/Zope-2.3.0-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: retrieve_messages) File /home/morten/Zope-2.2.4-src/lib/python/Products/GUM/gum.py, line 579, in retrieve_messages (Object: CatalogAware) File /home/morten/Zope-2.2.4-src/lib/python/Products/GUM/gum.py, line 655, in _retrieve_messages (Object: CatalogAware) File /home/morten/Zope-2.2.4-src/lib/python/Products/GUM/gum.py, line 986, in drop_message (Object: CatalogAware) File /home/morten/Zope/lib/python/Products/RFC822Message/util.py, line 86, in store_message (Object: CatalogAware) File /home/morten/Zope/lib/python/Products/RFC822Message/rfc822_message.py, line 66, in manage_add_rfc822_message (Object: CatalogAware) File /home/morten/Zope-2.3.0-src/lib/python/OFS/ObjectManager.py, line 285, in _setObject (Object: CatalogAware) File /home/morten/Zope/lib/python/Products/BTreeFolder/BTreeFolder.py, line 187, in _setOb (Object: CatalogAware) TypeError: object does not support item assignment """ *sound of headscratching* -Morten
On 24 Feb 2001, Morten W. Petersen wrote:
I've tried subclassing BTreeFolder, but then, whenever the object is accessed, zope falls flat on its face. :-\
Hm. That's usually a sign of a method being called that doesn't exist in my experience.
Morten W. Petersen wrote:
[Erik Enge]
| Can't you just subclass the BTree Folder as you would with OFS.Folder? | | I think you might be confusing the Zope BTree implementation with the | BTree Folder Product?
I've tried subclassing BTreeFolder, but then, whenever the object is accessed, zope falls flat on its face. :-\
If you're deriving from BTree, make sure that the class BTree's __init__ method gets called. Otherwise, there will be no BTree obeject to store your data in. Does your class have an __init__ method? If so, you need to put as its first line: MyClass.inheritedAttribute('__init__')(self) Where MyClass is the name of your class. -- Steve Alexander Software Engineer Cat-Box limited
participants (3)
-
Erik Enge -
morten＠esol.no -
Steve Alexander