[ZODB-Dev] ZODBAnalyze - simple ZODB analysis tool

Ivo van der Wijk ivo@amaze.nl
Sun, 22 Sep 2002 13:40:43 +0200


Hi All,

We have a 5.5 GB ZODB over here (sometimes growing to over 7GB if packing
fails a couple of days), and I wanted to know what's actually using all
the space in the ZODB. To find this out, I hacked ZODB3 3.1b1's fsdump a 
bit and created ZODBAnalyze. It will print counts for specific transaction
types, counts for objects types and their respective total sizes.

It will print reports like:

[ivo@1979 ZODBAnalyze]$ ./analyze.py ~/Zope/Zope-2.4.3-src/var/Data.fs
Transactions in /home/ivo/Zope/Zope-2.4.3-src/var/Data.fs: 411
Records in /home/ivo/Zope/Zope-2.4.3-src/var/Data.fs: 11286
Transactions sorted by size:
      size      count    average id
----------+----------+----------+--
  85942375          5   17188475 PUT
   2359431         29      81359 <unknown>
   2249387          3     749795 manage_addPhoto
   1578217         42      37576 bj_edit
    514499         29      17741 invokeFactory
    292147         13      22472 view
    192084         15      12805 index_html
    172721         19       9090 manage_delObjects
              :
Objects sorted by size
      size      count    average id
----------+----------+----------+--
  88741502        699     126954 OFS.Image.Pdata
    894895        850       1052 BTrees.IOBTree.IOBTree
    575361         29      19840 OFS.Image.Image
    496693       4753        104 BTrees.IIBTree.IISet
    432266        389       1111 BTrees.IOBTree.IOBucket
    396208         35      11320 Products.Photo.PhotoImage.PhotoImage
              :

You can find the code at:

http://vanderwijk.info/python/ZODBAnalyze/

Please read the comments in the source.

The tool isn't meant to give you exact figures, but an estimate on what
the space is used for, etc.

As far as I know, it should not corrupt databases (even when using it on
a running Zope instance), but still, backup and use at your own risk.

Cheers,

	Ivo

-- 
Drs. I.R. van der Wijk                                      -=-              
Brouwersgracht 132                             Amaze Internet Services V.O.F.
1013 HA Amsterdam, NL                                       -=-              
T +31-20-4688336         F +31-20-4688337        Linux/Web/Zope/SQL/MMBase   
W http://www.amaze.nl    E info@amaze.nl             Network Solutions        
W http://vanderwijk.info E ivo@amaze.nl                 Consultancy          
PGP http://vanderwijk.info/pgp                              -=-