Kapil Thangavelu wrote:
Shane Hathaway wrote:
BTW has anyone done any semi-formal speed tests? It's rumored that a custom getattr in Python can cause a bit of a slowdown. We could write a custom getattr in C to solve the problem. But we don't want to do any premature optimization.
Shane
I'll work on some tests.
Kapil
semi-formal test i ran some basic tests comparing a btree folder against a folder. i'm not sure about my testing methodology but here are the results. the test code is at the end of this email. the first pair of numbers is a timed creation test for size xxx of DTML Documents. the second pair of numbers is a timed test for len(objectIds()) the third pair of numbers is a timed test for attribute access with a call to int(dtmldoc.title_or_id()) # all the ids are stringed numbers my test box is running mandrake 7.1 with 512 M ram, dual proc celerons 550s (oc'd) some quick conclusions. the btree folder is slower on average for creation by anywhere from 15-25% andt btree's __getattr is on average almost twice as slow as the folder's getattr. order folder followed by btreefolder for size 100 0.0974419116974 :::0.126843929291 0.000378012657166 :::0.000354051589966 0.0101770162582 :::0.0186030864716 for size 100 0.0978569984436 :::0.125252962112 0.000376105308533 :::0.000353932380676 0.0101470947266 :::0.0185579061508 for size 500 0.52112698555 :::0.657492995262 0.00154399871826 :::0.00154197216034 0.0524510145187 :::0.0956329107285 for size 500 0.53660595417 :::0.714686989784 0.00154197216034 :::0.00157594680786 0.0532569885254 :::0.100463986397 for size 1000 1.11411702633 :::1.40103697777 0.00302505493164 :::0.00306105613708 0.105547070503 :::0.193196058273 for size 1000 1.12388503551 :::1.58762395382 0.00370097160339 :::0.00381696224213 0.106634020805 :::0.194367051125 for size 2000 2.60056698322 :::3.20675897598 0.007483959198 :::0.00741696357727 0.216197967529 :::0.396993994713 for size 2000 2.64964604378 :::3.26421093941 0.0074850320816 :::0.00761103630066 0.214147925377 :::0.40997505188 for size 5000 9.50434195995 :::11.0002789497 0.0184400081635 :::0.018413901329 0.533676981926 :::0.997251987457 just to be fair i reversed the positions and did a second set of runs with only 1 run per size as the results were consistent with the first set. for size 100 0.128319978714 :::0.10124707222 0.000413060188293 :::0.000383019447327 0.018816113472 :::0.0104240179062 for size 100 0.12813603878 :::0.101204037666 0.00040602684021 :::0.000380039215088 0.0188159942627 :::0.0105849504471 for size 500 0.681550979614 :::0.546862006187 0.00152194499969 :::0.00160706043243 0.0983099937439 :::0.0540620088577 for size 1000 1.47808301449 :::1.17385303974 0.00308799743652 :::0.00319290161133 0.197283029556 :::0.108732938766 for size 5000 11.1204429865 :::9.15097498894 0.015163064003 :::0.0150960683823 1.00954806805 :::0.535046935081 Despite this evidence. at size 5000, looking at the btreefolder contents in the management screen was near instaneous while the folder took almost a minute. based on the above evidence though that has more to do with the interface than the underlying speed of the folder methods. night. Kapil test code to follow. external method import time import os import Products.BTreeFolder def btest(self, REQUEST, MMMM=None): ''' ''' if not MMMM: MMMM=100 results = [] bf = Products.BTreeFolder.BTreeFolder.BTreeFolder() bf.id = 'btree_test_b' self._setObject(bf.id, bf) id = 'btree_test_f' rf = self.manage_addFolder(id) # creation test for completeness stime = time.time() for num in xrange(MMMM): self.btree_test_b.manage_addDTMLDocument(str(num)) etime = time.time() result = etime-stime #self.get_transaction().commit() stime = time.time() for num in xrange(MMMM): self.btree_test_f.manage_addDTMLDocument(str(num)) etime = time.time() results.append(result, etime-stime) # access test stime = time.time() len(self.btree_test_b.objectIds()) etime = time.time() result = etime-stime stime = time.time() len(self.btree_test_f.objectIds()) etime = time.time() results.append(result, etime-stime) # iteration test stime = time.time() for oid, item in self.btree_test_b.objectItems(): int(item.title_or_id()) etime = time.time() result=etime-stime stime = time.time() for oid, item in self.btree_test_f.objectItems(): int(item.title_or_id()) etime = time.time() results.append(result, etime-stime) return results