[Zope] Analyzing ZODB objects

[Dieter Maurer]

Bjorn Stabell wrote at 2003-10-23 09:14 +0800:
 > > From: Dieter Maurer [mailto:dieter at handshake.de] 
 > > Bjorn Stabell wrote at 2003-10-22 15:53 +0800:
 > >  > What is a good tool for analyzing the size of each object 
 > > > in the ZODB,  > pickled and unpacked in memory?  I'm using Zope
 > 2.6.2.
 > > 
 > > I do not think there is such a tool.
 > > 
 > > I enhanced the output of "fsdump" to include the pickle size 
 > > of object records in a transaction. The main purpose has been 
 > > to analyse the reasons for unexpected transaction sizes 
 > > (turned out to be ZCatalog Metadata).
 > 
 > Would it be possible to deduct anything about the runtime memory
 > footprint of python objects based on their pickle size?

They can be far apart. Although, when your pickle is several MB
your object is not several bytes and vice versa.


 > I'm just dreaming here; a tool that would be useful in analyzing and
 > optimizing performance of Zope servers, as well as answering questions
 > such as "what the heck is in that ZODB?" :)  Something like the database
 > administration tools you have for RDBMS that allow you to browse and
 > change them.

But you can browse and change the ZODB. The tool is called the ZMI.
I do not think you should try this on the raw ZODB level
(the ZODB is a mapping of object ids to objects without any structure;
you do not want to work on this level).

 > For the ZODB, I guess this kind of tool could:
 > 
 > - let you browse the database objects, hieararchical tree-like view,
 > perhaps without understanding classes at all, just treating objects as
 > containers of other classes and/or basic python data types.  kind-of
 > like a file system view that:
 > 	- shows the types of objects
 > 	- shows the contents of objects with basic types
 > 	- shows the size of objects and total for its subobjects
 > 	- lets you edit the content types etc

This would be a small extension of the ZMI.

The ZMI already does:

  *  show the type of objects (visualizes them as icons)

  *  shows the content of objects as far as the content are
     properties. You see the type and the value and can change
     the value

  *  sizes are only shown for some objects (and object overhead is
     not counted)

  *  do not know what you mean with "content type" but you
     can edit the content.
  

 > - lets you browse transactions (like your fsdump?), single or multiple
 > at a time, and objects changed in transactions in a similar way to above
 > (excluding option to edit objects)

The ZODB currently knows a single use case for browsing transactions:
selecting transactions that should be undone.

What use case do you have in mind.

Unless, I am analysing strange behaviour, I am not interested
in browsing transactions.

 > This tool could be standalone, or run through Zope.
You can use the ZMI (more precisely its actions) from a script
(without accessing Zope through HTTP).

 > Is this feasible?

I do not yet understand why you would want such a thing.
Can you provide use cases?


Dieter