Thanks Dylan. I'll try this out. It 'looks' right for what I'm looking to do. Now to figure out the details... I'll have an umbrella script somewhere, triggered by an event, not sure what, to do the updates. Another suggestion made by 'zerkani_user_list' (name on email) is that I store the data to be published in a database, like a summary table in a DW, rather than the HTML code. I am looking into this approach as an alternative, but I feel storing the HTML is more suitable to the business logic in this instance. Ultimately it'll come down to a combination of suitability & convenience. Thanks to everyone. Regards, Samir. -----Original Message----- From: Dylan Reinhardt [mailto:zope@dylanreinhardt.com] Sent: Wednesday, May 28, 2003 19:05 To: Samir Mishra Cc: zope@zope.org Subject: RE: [Zope] Storing published objects (?) What you're describing could be done fairly easily with a Python product. ------ # untested, top-of-head, etc. class DeepCache(REGULAR_MIXINS): def __init__(self, id, title, object_path): self.id = id self.title = title self.obj_path = object_path self.results = self.refresh() index_html = DTMLFile('index', globals()) # points to a dtml file that consists of: # <dtml-var view> def view(self): return self.results def refresh(self): self.results = self.restrictedTraverse(object_path) ------ That's an incomplete product (no security, no constructor method, etc), but it's a decent sketch of a general-purpose object that will allow you to cache the output of any object you can put in the ZMI. This, of course, assumes that you want caching refreshed manually... automatically detecting when underlying data has changed is quite a bit more difficult and must be done in a target-specific way. If you're not updating too often, an approach like this should work fine... if your objects change with any frequency, you'll want to make sure to pack the ZODB pretty aggressively. HTH, Dylan On Tue, 2003-05-27 at 23:56, Samir Mishra wrote:

Jamie,

Thanks for the response. I think you misunderstood my question. I'm not looking to implement a cache manager.

What I'm looking to do is, not store the page for a few minutes/hours, but months. Or longer. I also want external apps to have access to the rendered page.

I have to generate weekly financial reports. The data for each report is from NUMEROUS sources. So the query, formatting etc. are complex and time consuming. The weekly report gets updated over the duration of the month, but at the end of the month the report is frozen. So once the report is published, it will almost NEVER be changed. We may publish corrections to the report, but these will be labelled as corrections, and will not replace the originally published document.

I'd rather not have to re-create the report every time it's viewed. I'd prefer to save the HTML representation of the report as a BLOB in a database (and later extend this to PDFs of the same). And that's what I'm hoping to do - assign the results of a page rendered by Zope, i.e., all of the HTML generated, to a binary column in a DB.

OT: I believe Berkeley DB is extremely fast for this sort of indexing/retrieval. Anyone have any experience?

Hope that's clearer.

Thanks again.

Samir.

-----Original Message----- From: Jamie Heilman [mailto:jamie@audible.transient.net] Sent: Wednesday, May 28, 2003 08:25 To: Samir Mishra Cc: zope@zope.org Subject: Re: [Zope] Storing published objects (?)

Samir Mishra wrote:

...

The underlying SQL queries are complex, the data doesn't change often, and I think this may be the best way to lighten the burden on the DB.

Hmmm, well generally this is what CacheManagers do, but storing the results in a database would be a new one (to date I've seen managers that use memory, the filesystem, and just add HTTP headers to the response). While its entirely likely you could achieve what you want using a memory cache, if the data is sufficiently large it may be not be a good idea, and its certainly not persistent accross zope-restarts. Implementing a cache manager that stored its data in a RDBMS shouldn't be impossible, take a look at the Cache & CacheManager interfaces in OFS/Cache.py -- and I'd suggest using the updated code I've prepared in http://collector.zope.org/Zope/911 as its cleaner than the existing stuff.