[Zope] Analyzing ZODB objects

[Bjorn Stabell]

> From: Dieter Maurer [mailto:dieter at handshake.de] 
[...]
> They can be far apart. Although, when your pickle is several 
> MB your object is not several bytes and vice versa.

Well, in that case it might be useful for such a ZODB admin tool to show
both sizes.  It could be a combined cache analysis and ZODB browsing
tool.


[...]
> I do not yet understand why you would want such a thing.
> Can you provide use cases?

I guess I want something very low-level, for use in debugging strange
behavior, and for help in understanding how Zope apps are built.  The
ZMI works with object interfaces, which is useful, but requires that
each object supports an interface (ObjectManager etc).  Many objects
don't, especially not when you're developing them :)  For this
reason--no access to data except through application-provided
interfaces--ZODB feels much like a "black box" to me.

Example use cases include:

- SPACE AND MEMORY OPTIMIZATION. Reducing ZODB size, and Zope memory
usage.  I've got some huge objects in my database, which ones are they?
Why are they huge?  If I know this, I can optimize.  (Related: My Zope
uses a lot of memory; why?  Objects of which class, and in which
location, uses the most memory?  Why were they loaded? At the ZMI level,
you don't want to know if objects are loaded/ghosted.)

- UNDERSTANDING CONTENT TYPES AND TOOLS.  What is the difference in the
data structure of PloneDocument and CMFDocument?

- DEBUGGING CLASS MIGRATION PROBLEMS.  Some older objects are exhibiting
strange behavior; what is the difference in data structure between them
and the new objects?  Which objects of class X doesn't have attribute A
set?  (Except for the obvious Zope/CMF/Plone/Product upgrades, these
kinds of problems happen a lot during development each time a class is
changed without recreating its objects.)

- VERIFYING AND IMPROVING DATA STORAGE SCHEMAS.  Does class X really
store the attributes I thought it would?

- CHANGING A PROPERTY THAT DOESN'T HAVE A MANAGEMENT INTERFACE.  For
debugging, testing, or migration purposes, or for just fixing a one-off
bug.


The truth is, I think this kind of tool will "open my eyes" to what's in
the ZODB and take much of the guesswork out of developing with Zope,
similar to the eye opening experience a RDBMS admin tool is.  I often
write scripts to analyze object structure and do simple changes; I wish,
however, that there would be an admin environment that provided:

- A database browsing / object inspector tool, taking away the need to
write scripts for browsing/changing objects in most cases, and
encouraging people to analyze and understand the database structure

- A query language that would make it even easier to write scripts (+
ZODB index support?)

- A place to store one-off scripts so they don't get mixed up with the
application

Something like this http://www.pgadmin.org/pgadmin3/screenshots.php, but
for the ZODB.


Regards,
-- 
Bjorn