[ZODB-Dev] Wrong blob returned in one of the zeo clients

[Shane Hathaway]

On 03/01/2011 08:05 AM, Maurits van Rees wrote:
> Op 01-03-11 13:54, Maurits van Rees schreef:
>> Op 01-03-11 04:41, Shane Hathaway schreef:
>>> On 02/28/2011 09:29 AM, Maurits van Rees wrote:
>>>> This is now also happening for some images during normal operation, so
>>>> after any blob migration has been run and existing blob caches have been
>>>> cleared.  What happens is that somehow the file contents for
>>>> 0xblahblah.blob in the blob caches can differ between two zeo clients,
>>>> even when the image has not changed at all (only tried with images as it
>>>> is easier to spot than with other files).
>>>
>>> Thanks for the report.  Looking through the blob code, I noticed that
>>> when the system is using a blob cache, it was possible for a thread to
>>> get a partial file while another thread is downloading that file.  I
>>> suspect this is the problem you saw.  I have committed a fix.
>>
>> Thanks!  I'll test this.
>
> No, I am still seeing it.  I now see more clearly that the problem is
> that two files share the same blob name.  I completely remove the blob
> cache and restart the zeo client.  I visit image number 1 and get this
> file in the var/blobcache:
> 291/0x038c599924e70177.blob
>
> I then visit the second image and get served the same file!  Now I
> remove this file from the blob cache, load the second image in a browser
> and this time I get the correct content for this image, using the same
> file name in the var/blobcache directory.  Now of course loading the
> first image gives the wrong content.
>
> The .layout file is set to 'zeocache'.
>
> Any idea where the error might be?  Could this be in plone.app.blob?
> Any chance that this is better in ZODB 3.9+?

This appears to be a design bug in the "zeocache" blob layout: if a 
single transaction stores multiple blobs, and the OIDs of some of those 
blobs differ by a multiple of 997, then cache filenames will collide.  I 
think the solution is to incorporate the OID, not just the transaction 
ID, in the cache filename.  If I have diagnosed this correctly, this 
will have to be solved in ZEO as well.

Nice catch!

Shane