Storing changes in ZODB, without keeping old versions
The ZCache Product I created has one major problem - every time it updates itself it'll make the ZODB grow, since it stores all the old contents it had. I don't want this to happen - there's no reason why anyone should undo a change to the cache. I do however want the contents of the ZCache to be stored on disk when it gets moved out of the Zope cache, so I don't want to use volatile variables. Is there any way to do this? Or is it impossible? -- Itamar S.T. itamars@ibm.net
On Thu, 25 Nov 1999, Itamar Shtull-Trauring wrote:
The ZCache Product I created has one major problem - every time it updates itself it'll make the ZODB grow, since it stores all the old contents it had. I don't want this to happen - there's no reason why anyone should undo a change to the cache. I do however want the contents of the ZCache to be stored on disk when it gets moved out of the Zope cache, so I don't want to use volatile variables.
Is there any way to do this? Or is it impossible?
If you don't want it in memory, and you don't want it in a SQL backend, you'll need to store the contents in a file in Zope's var directory. -- ___ // Zen (alias Stuart Bishop) Work: zen@cs.rmit.edu.au // E N Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au //__ Computer Science, RMIT WWW: http://www.cs.rmit.edu.au/~zen
At 09:05 PM 11/25/99 , Itamar Shtull-Trauring wrote:
The ZCache Product I created has one major problem - every time it updates itself it'll make the ZODB grow, since it stores all the old contents it had. I don't want this to happen - there's no reason why anyone should undo a change to the cache. I do however want the contents of the ZCache to be stored on disk when it gets moved out of the Zope cache, so I don't want to use volatile variables.
Is there any way to do this? Or is it impossible?
Why would you want the cache to be persistent? The data is already elsewhere in the ZODB, and the usual behaviour of a cache is to not persist over reboots. The ZODB at this moment doesn't support mixed FileStorages, which would be the way to store both versioned and non-versioned persistent objects. You'll have to make your own storage in the var directory. -- Martijn Pieters, Web Developer | Antraciet http://www.antraciet.nl | Tel: +31-35-7502100 Fax: +31-35-7502111 | mailto:mj@antraciet.nl http://www.antraciet.nl/~mj | PGP: http://wwwkeys.nl.pgp.net:11371/pks/lookup?op=get&search=0xA8A32149 ------------------------------------------
Martijn Pieters wrote:
Why would you want the cache to be persistent? The data is already elsewhere in the ZODB, and the usual behaviour of a cache is to not persist over reboots.
1. Expensive methods - one use for the zcache is storing expensive methods' results without having to run them every time you need their info. 2. Lets say you want an article that changes once an hour (or the output of fortune, like Slashdot has). Write a method that outputs the next article in the list of possible articles every time it's run, and the have a zcache point at this method with a expiration rate of 1 hour. Now your article gets updated every hour, assuming its being read - if no-one reads it, it isn't updated, so you don't waste content, and you don't have to worry about cron jobs. I might add an alternative expiration method, maybe similar to the cron syntax, so you can have it expire at a certain hour. For example, expire cache contents at midnight every day would update your article once a day, as opposed to current method which limits you stuff like expiring if 24 hours have passed since the last access. On an unrelated note, writing the contents of the ZCache to file instead of using the ZODB has the benefit that I can use the same method Apache uses for generating Etag headers - inode together with last-modified date. (Etag headers are used by HTTP proxies as a unique identifier for your page so that they can check when the content on the web server has changed.) So I'll probably end up using files loaded by __setstate__ into volatile variables. -- Itamar S.T. itamars@ibm.net
On Fri, 26 Nov 1999, Itamar Shtull-Trauring wrote:
1. Expensive methods - one use for the zcache is storing expensive methods' results without having to run them every time you need their info.
For the same reason I hacked up tmpStorage which stores the results of expensive methods on the filesystem. It is at: http://www.zope.org/Members/gaaros/localcookies
So I'll probably end up using files loaded by __setstate__ into volatile variables.
Is this for a read only application? ie what happens when you make change? Do you want it to propagate to the filesystem? Pavlos
It sounds like the problem that would come from storing session data in ZODB. One nice solution (as someone pointed out for me) would be to have multiple parallell ZODB-storages. One for the normal transactional ZODB(filebased or RDMS-based) and one non-transactional for session-data and cache-data. Both would be distributed by the ZEO for failover and scalability. Regards, Johan Carlsson
The ZCache Product I created has one major problem - every time it updates itself it'll make the ZODB grow, since it stores all the old contents it had. I don't want this to happen - there's no reason why anyone should undo a change to the cache. I do however want the contents of the ZCache to be stored on disk when it gets moved out of the Zope cache, so I don't want to use volatile variables.
Is there any way to do this? Or is it impossible?
-- Itamar S.T. itamars@ibm.net
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev No cross posts or HTML encoding! (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
participants (5)
-
Itamar Shtull-Trauring -
Johan Carlsson -
Martijn Pieters -
Pavlos Christoforou -
Stuart 'Zen' Bishop