[Zope3-Users] getting random results out of a catalogs field index

[Christian Theune]

Hey,

Am Samstag, den 05.05.2007, 19:27 +0200 schrieb Dominique Lederer:
> hi, thanks for the reply, i just managed to improve the performance of my query
> significantly:
> 
> what i wanted to do was:
> 
> - retrieve the len() of the catalog index
> - retrieve a list() of the Resultset
> - accessing n random results and their objects
> 
> to retrieve a random object i did:
> 
> query = catalog.apply({'myIndex':(None,None)})
> length = len(query)
> index_intids = list(query)
> intid = all[random.randint(0,len_all-1)]
> object = getObject(intid)
> 
> which was with 10000 items in the index slow (i had to wait 2-3 seconds for a
> view to render)
> 
> after looking into the field index implementation i changed the above lines to:
> 
> length = len(catalog['myIndex']._rev_index)
> index_intids = list(catalog['myIndex']._rev_index.keys())
> 
> which now works like a charm.

Hmm. But it's at least not memory efficient and it has to load the whole
reverse index, at least the keys. Given those are integers 10k elements
are probably not that much of a problem. However, you're probably
screwing up your LRU caches as a 10k BTree needs to load quite a few
nodes and buckets.

> i am not an expert with BTrees so i cant really say what the problem is/was.

Well, essentially you have cut out the searching.

Christian

-- 
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope3-users/attachments/20070507/fe00454d/attachment.bin