[ZODB-Dev] BTree memory bomb

[Simon Burton]

Hi all,

I did a test (below) to see if BTree would unload it's objects as it grew large. No luck, I
killed the script once it had taken 80% of memory. 

How do I make a mapping object that can grow arbitrarily large (limited by disk space) ?
This is primarily why I chose ZODB.

thanks for any help!

Simon.


from ZODB import FileStorage, DB
from BTrees.OOBTree import OOBTree

from time import sleep

def main():
  storage=FileStorage.FileStorage('test.fs')
  db = DB(storage, cache_size=400)
  connection = db.open()
  root = connection.root()

  data = OOBTree()
  
  root[0] = data
  print "data:", len(data)

  f = open('/dev/zero')

  for i in xrange(10000):
    for j in xrange(10000):
      data[i*10000+j] = f.read(i*128)
    get_transaction().commit()
    print "data:", len(data)

  print "sleep"
  sleep(100)

############################

Here are the file sizes:

blue:~/home/python/lang$ ls -lh test*
-rw-r--r--  1 simon simon 403M Jan 18 14:36 test.fs
-rw-r--r--  1 simon simon 119K Jan 18 14:34 test.fs.index
-rw-r--r--  1 simon simon    5 Jan 18 14:33 test.fs.lock
-rw-r--r--  1 simon simon  31M Jan 18 14:36 test.fs.tmp
-rwxr-xr-x  1 simon simon  717 Jan 18 14:33 test.py


-- 
Simon Burton, B.Sc.
Licensed PO Box 8066
ANU Canberra 2601
Australia
Ph. 61 02 6249 6940
http://arrowtheory.com