[ZODB-Dev] zeo.memcache

[Laurence Rowe]

On 12 October 2011 23:53, Shane Hathaway <shane at hathawaymix.org> wrote:
> As I see it, a cache of this type can take 2 basic approaches: it can
> either store {oid: (state, tid)}, or it can store {(oid, tid): (state,
> last_tid)}. The former approach is much simpler, but since memcache has
> no transaction guarantees whatsoever, it would lead to consistency
> errors. The latter approach makes it possible to avoid all consistency
> errors even with memcache, but it requires interesting algorithms to
> make efficient use of the cache. I chose the latter.

On first reading I had thought that the {oid: (state, tid)} approach
would not necessarily lead to consistency errors as a connection could
simply discard cached values where the cached state tid is later than
the current transaction's last tid. But I guess that it must be
impossible for a committing connection to guarantee that all cached
oids remain invalidated during a commit and are not refilled with a
previous state by another connection performing a read. This would
necessitate the same checkpointing algorithm to avoid consistency
errors.

I sometimes wonder if it would be better to separate the maintenance
of the oid_tid mapping from the storage of object states. A database
storing only the oid_tid mapping and enough previous tids to support
current transactions -- essentially the Data.fs.index -- would always
fit easily in RAM and could conceivably be replicated to every machine
in a cluster to ensure fast lookups. The storage / caching of object
states could then be very simple.

Laurence