[ZODB-Dev] strange savepoint behavior

[Tim Peters]

[dvd]
> ...
> I'm using ZODB 3.5.1

OK.

[Tim]
>> ..
>> If I add
>>
>>     transaction.commit()
>>
>> right after your
>>
>>     data = root['data'] = OOBTree()
>>
>> then the POSKeyError goes away, and I get
>>
>>     Traceback (most recent call last):
>>       File "pos_dvd.py", line 34, in ?
>>         print commonWords[1].id
>>     AttributeError: 'Word' object has no attribute 'id'
>>
>> instead ...

> great! this is exactly the error that i'm trying to reproduce so is this
> an expected behavior (the lost of the attributes)?

Yes.

> this sounds a bit strange to me, let me explain:
>
> with:
> **** CODE ****
> sv = transaction.savepoint()
> for word in commonWords:
> 	sv2 = transaction.savepoint()
> 	data[word.id] = word
> **************
>
> the last line (print ....) raise an AttributeError:
>
> with:
> **** CODE ****
> sv = transaction.savepoint()
> for word in commonWords[0:1]:
                         ^^^^^

I think that (^^^^^) is the only change you made from above.
 
> 	sv2 = transaction.savepoint()
> 	data[word.id] = word
> **************
>
> all works fine (no AttributeError)
>
> obviously if i omit the second savepoint, all works fine

That's because the transaction machinery never learns about any of your Word
objects than.

> if rolling back invalidates the in-memory objects, why without the second
> savepoint the "id" attribute still exists?

I could <wink> have been more precise:  it's the in-memory objects the
transaction _knows_ about.  None of this makes sense without context, so
let's start over:

"""
commonWords = []
count = "0"
for x in ('hello', 'world', 'how', 'are', 'you'):
        commonWords.append(Word(x, count))
        count = str(int(count) + 1)
"""

At this point, none of the Word instances are reachable from the database
root, and the transaction machinery knows nothing about them.  They're just
floating in RAM, with no connection at all to the database.  For the
transaction machinery to learn about a new object, (a) it has to become
reachable from the root object; and, (b) a commit or savepoint needs to be
done.  Until those things happen, new objects live purely in RAM, and the
transaction machinery literally doesn't know they exist.

"""
sv = transaction.savepoint()
"""

Still no connection to the database.  All this captures is the empty BTree
attached to the root by the earlier:

"""
data = root['data'] = OOBTree()
"""

Now your loop:

"""
for word in commonWords[0:1]:
 	sv2 = transaction.savepoint()
 	data[word.id] = word
"""

This loop goes around exactly once.  It first does a savepoint, but nothing
has changed since the

    sv = transaction.savepoint()

before the loop.  All of the Word objects still have no connection to the
database at this point, and the transaction machinery still knows nothing
about them.  When the loop does

	data[word.id] = word

then commonWords[0] becomes reachable from the root, but no savepoint or
commit is done, so the transaction machinery _still_ doesn't know anything
about commonWords[0].  Then

    sv.rollback()

Here you're rolling back to the point where the BTree was empty, and if you
added

    print list(data)

you'd see

    []

in the output.  Nothing else got rolled back, because the persistence
machinery never learned about anything else.  So

    print commonWords[1].id

still "worked", and, indeed, so would

    print commonWords[0].id

Now change your loop to

    for word in commonWords[0:2]:

Now the loop goes around twice, and on the _second_ time through the loop
the

	sv2 = transaction.savepoint()

"sees" that commonWords[0] is now reachable from the root, and saves away
its current state.  The transaction machinery went from knowing nothing, to
knowing everything, about commonWords[0], because this savepoint got made
and the previous loop iteration made commonWords[0] reachable from the root.

When you later do

    sv = transaction.savepoint()

you're explicitly asking the transaction machinery to throw away _all_
current state _that it knows about_.  This now includes the state for
commonWords[0], but not for any other Word instance.

So the rollback throws away the state for commonWords[0], and a later

    print commonWords[0].id

would raise

    AttributeError: 'Word' object has no attribute 'id'

However, the transaction machinery still doesn't know anything about any
other Word object, so

    print commonWords[i].id

will not raise AttributeError for any i > 0.  Since the transaction
machinery doesn't know those exist, it doesn't touch them one way or the
other.

In general, if you wrote the loop head as

    for word in commonWords[0:n]:

then the savepoints inside the loop would teach the transaction machinery
that the Words in commonWords[0:n-1] exist, and a later

    print commonWords[i].id

will raise AttributeError if and only if i < n-1.

But those details are all artifacts of exactly how your test case is
written.  As a general rule, if you do a rollback _beyond_ the creation of a
new persistent object P, you shouldn't expect P to be in a useful state
anymore (and, as you've seen, the rollback will destroy P's state if the
transaction machinery knows about P at the time the rollback is done -- and
if P came into existence after the savepoint to which the rollback was done,
there's no record remaining anywhere of what P's state was before the
rollback).