[Zope] Storing published objects (?)

[Samir Mishra]

Thanks Dylan. I'll try this out. It 'looks' right for what I'm looking to
do. Now to figure out the details...

I'll have an umbrella script somewhere, triggered by an event, not sure
what, to do the updates.

Another suggestion made by 'zerkani_user_list' (name on email) is that I
store the data to be published in a database, like a summary table in a DW,
rather than the HTML code. I am looking into this approach as an
alternative, but I feel storing the HTML is more suitable to the business
logic in this instance. Ultimately it'll come down to a combination of
suitability & convenience.

Thanks to everyone.

Regards,
Samir.


-----Original Message-----
From: Dylan Reinhardt [mailto:zope@dylanreinhardt.com]
Sent: Wednesday, May 28, 2003 19:05
To: Samir Mishra
Cc: zope@zope.org
Subject: RE: [Zope] Storing published objects (?)


What you're describing could be done fairly easily with a Python
product.

------

# untested, top-of-head, etc.


class DeepCache(REGULAR_MIXINS):

    def __init__(self, id, title, object_path):
        self.id = id
        self.title = title
        self.obj_path = object_path
        self.results = self.refresh()

    index_html = DTMLFile('index', globals())
    # points to a dtml file that consists of:
    # <dtml-var view>

    def view(self):
        return self.results

    def refresh(self):
        self.results = self.restrictedTraverse(object_path)

------


That's an incomplete product (no security, no constructor method, etc),
but it's a decent sketch of a general-purpose object that will allow you
to cache the output of any object you can put in the ZMI.  This, of
course, assumes that you want caching refreshed manually...
automatically detecting when underlying data has changed is quite a bit
more difficult and must be done in a target-specific way.

If you're not updating too often, an approach like this should work
fine... if your objects change with any frequency, you'll want to make
sure to pack the ZODB pretty aggressively.

HTH,

Dylan

    



On Tue, 2003-05-27 at 23:56, Samir Mishra wrote:
> Jamie,
> 
> Thanks for the response. I think you misunderstood my question. I'm not
> looking to implement a cache manager.
> 
> What I'm looking to do is, not store the page for a few minutes/hours, but
> months. Or longer. I also want external apps to have access to the
rendered
> page. 
> 
> I have to generate weekly financial reports. The data for each report is
> from NUMEROUS sources. So the query, formatting etc. are complex and time
> consuming. The weekly report gets updated over the duration of the month,
> but at the end of the month the report is frozen. So once the report is
> published, it will almost NEVER be changed. We may publish corrections to
> the report, but these will be labelled as corrections, and will not
replace
> the originally published document. 
> 
> I'd rather not have to re-create the report every time it's viewed. I'd
> prefer to save the HTML representation of the report as a BLOB in a
database
> (and later extend this to PDFs of the same). And that's what I'm hoping to
> do - assign the results of a page rendered by Zope, i.e., all of the HTML
> generated, to a binary column in a DB.
> 
> OT: I believe Berkeley DB is extremely fast for this sort of
> indexing/retrieval. Anyone have any experience?
> 
> Hope that's clearer.
> 
> Thanks again.
> 
> Samir.
> 
> -----Original Message-----
> From: Jamie Heilman [mailto:jamie@audible.transient.net]
> Sent: Wednesday, May 28, 2003 08:25
> To: Samir Mishra
> Cc: zope@zope.org
> Subject: Re: [Zope] Storing published objects (?)
> 
> 
> Samir Mishra wrote:
> > The underlying SQL queries are complex, the data doesn't change often,
and
> I
> > think this may be the best way to lighten the burden on the DB.
> 
> Hmmm, well generally this is what CacheManagers do, but storing the
> results in a database would be a new one (to date I've seen managers
> that use memory, the filesystem, and just add HTTP headers to the
> response).  While its entirely likely you could achieve what you want
> using a memory cache, if the data is sufficiently large it may be not
> be a good idea, and its certainly not persistent accross
> zope-restarts.  Implementing a cache manager that stored its data in a
> RDBMS shouldn't be impossible, take a look at the Cache & CacheManager
> interfaces in OFS/Cache.py -- and I'd suggest using the updated code
> I've prepared in http://collector.zope.org/Zope/911 as its cleaner
> than the existing stuff.