[Zope] How can I get a listing of all objects in the Zope 2.62 ZODB?

[Chris Withers]

Mark, Jonathan (Integic) wrote:
> has been a positive one. I am grateful to Chris for writing it and am
> glad that he has moved on to other cutting edge Zope projects.

What gives you the idea I've "moved on" anywhere? I still use Squishdot 
heavily and deployed a new instance for a customer a couple of months ago...

> However, there is one problem. As the number of posts gets past 2000
> the application slows to a crawl.

That's not been my experience at all..

But, it does highlight one thing: you must be a developer - coming with 
a solution rather than a problem and asking a question that has little 
to do with either ;-)

The question I believe you meant to ask is "My squishdot site is slow, 
how can I speed it up?" rather than this stuff about zodb internals.

You could have even asked on the Squishdot mailing list ;-)

> You can observe the problem by comparing
> http://jonathanmark.com/Blog/1128110364/index_html, which is in a
> Squishdot instance with 3000 posts and is slow, and
> http://goodbyejim.com, which has 800 posts. The front page of
> jonathanmark.com/Blog is cached, hence one must visit an interior page
> to observe the problem. 

Caching can be a legitimate solution to these problems, though I've 
never needed it myself...

> All of these Squishdot instances run in the same Zope instance.

Okay, in this order:

- what version of Squishdot are you using?

- what version of Zope?

- what customisations have you done to the various templates?
   (I'd put money on something in here inadvertantly trying to drag all
    the posts into memory...)

- what size is your Data.fs?

- How much memory does the box have?

- What configuration changes have you made to Zope to ensure the object
   cache is of a useful size?

> So I decided that I would modify Squishdot so that it uses a
> relational database to store posts. 

"hey, this hammer's broken, gimme that wrench and I'll see if I can bash 
this nail in"

This is an awful lot of work for very little gain, why don't you 
actually try and solve the problem you're having rather than rewriting 
the entire app/framework/whatever?

> I am certain that through proper
> use of RDBMS indexes Squishdot would be scalable into the tens of
> thousands of messages. 

I'm certain that proper use of ZCatalog indexes, which modern Squishdots 
already use, will scale just as well ;-)

> For a small blog such as mine with several
> thousand posts a year even Gadfly might be good enough.

Hahahaha... ok, you made me laugh. Seriously though, don't even go 
anywhere near gadflu...

> To convert Squishdot to use an RDBMS I would have to modify the
> Posting and other product scripts in Squishdot. 

...which will take about 100 times more effort than figuring out what's 
slowing down your sites ;-)

> If I could look at the
> Squishdot-related objects in the ZODB I would have a better
> understanding of what Squishdot is doing when it posts and retrieves a
> message. 

No, you really wouldn't.

> If I may add my two cents on a related matter, what I like best about
> Zope 2 is the VHM, undo, and content management.

You loose undo with your plan, as well as incurring an awful lot more 
work...

> I work with RDBMSs
> every day and understand them far better than I do the ZODB. 

Really? It doesn't show ;-) In all fairness, yes, if you understand RDB 
more, then your temptation will naturally be to run to it, but that is a 
mistake here...

> more complex for DTML scripters like me. If one were to ask why Ruby
> on Rails and Django get the attention that Zope does not, I feel that
> a reliance on using the ZODB for textual transactional data is one
> reason.

I rather strongly disagree with this statement, but this isn't the right 
time or place to debate it...

> It sounds more difficult to set up an RDBMS app in Zope than in Django
> or Ruby on Rails.

Indeed, whereas it is trivial to set up a ZODB-based app with indexing 
done using ZCatalog.

> the hype. Since users prefer RDBMSs (that is what we know) new
> developers are going to gravitate towards Django or RoR.

They're welcome to it them...

> Wouldn't it be great if there could be a Zope app that would set up
> RDBMS tables the way Django does, or generate code to access tables
> the way RoR does? 

No. For me, it would suck slimy rocks through straws.
Now, a really good ORM that worked transparently with Zope's transaction 
management and allowed interaction in a similar style to zodb _would_ be 
of interest, but Zope 3 is your best bet for that...

> But for lack of a better idea in the meantime it is
> best for me to migrate my version of Squishdot to use an RDBMS.

I'm afraid I won't waste my time or yours helping you with that. If you 
want help with getting your sites faster, quickly, just answer the 
questions above...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk