[ZODB-Dev] blob todos

[Jim Fulton]

On Jun 8, 2007, at 12:36 PM, Christian Theune wrote:

> Hi,
>
> (this goes mostly to Jim)
>
> I just noticed that the blob.py has a todo list at the end of it. We
> shouldn't let it stay there. Here are some comments about the various
> entries:
>
> # To do:
> #
> # Production
> #
> #     - Ensure we detect and replay a failed txn involving blobs  
> forward
> or
> #       backward at startup.
> #
> #     Jim: What does this mean?
>
> Chris McDonough came up with this, I'm not quite sure. It has  
> something
> todo how FileStorage recovers when starting up AFAIR.

FileStorage doesn't play anything forward on startup.  It does throw  
away partial or uncommitted transactions.  I don't see any  
significant harm in leaving extra blob files around and they would  
eventually be removed through packing.   We do need to take a little  
extra care in the packing code to deal with this possibility though.

>
> # Far future
> #
> #       More options for blob directory structures (e.g. dirstorages
> #       bushy/chunky/lawn/flat).
> #
> #       Make the ClientStorage support minimizing the blob
> #       cache. (Idea: LRU principle via mstat access time and a
> #       size-based threshold) currently).
>
> I can put those into launchpad as blueprints.

+1

>
> # Savepoint support
> # =================
> #
> #  - A savepoint represents the whole state of the data at a certain
> point in
> #    time
> #
> #  - Need special storage for blob savepointing (in the spirit of
> tmpstorage)
> #
> #  - What belongs to the state of the data?
> #
> #    - Data contained in files at that point in time
> #
> #    - File handles are complex because they might be referred to from
> various
> #      places. We would have to introduce an abstraction layer to  
> allow
> #      switching them around...
> #
> #      Simpler solution: :
>
> Didn't you (Jim) do this?

Yes.  Non-optimistic savepoints now work AFAIK.  Interestingly, they  
were mostly implemented already.

Good catch wrt this to-do list.

Jim


--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org