[ZODB-Dev] Exception while closing ZODB

[Tim Peters]

[Ulla Theiss]
> since Version Zope 2.7.0 we have a problem closing the ZODB under
> Windows.
>
> Sometime the class LockFile (in the file .../ZODB/lock_file.py) throws
> while unlinking the lockfile the Exception:
> OSError: [Errno 13] Permission denied '....\\var\\...fs.lock'
>
> We open and close ZODB-databases in different threads.

That's dangerous; more below.

> A look at the sourcecode in LockFile.close() shows, that first
>    unlock_file(self._fp)
>    self._fp.close()
>
> is called. Which means other threads and processes are able to use the
> locked file again. Afterwards the lock-file should be deleted:
>
>    os.unlink(self._path)

Let's look at all the code:

class LockFile:
    def __init__(self, path):
        self._path = path
        try:
            self._fp = open(path, 'r+')
        except IOError, e:
            if e.errno <> errno.ENOENT: raise
            self._fp = open(path, 'w+')
        lock_file(self._fp)
        print >> self._fp, os.getpid()
        self._fp.flush()

    def close(self):
        if self._fp is not None:
            unlock_file(self._fp)
            self._fp.close()
            os.unlink(self._path)
            self._fp = None

As best I understand you:

    thread A created a LockFile, and later called its close() method

    simultaneously, thread B calls LockFile.__init__ with the same path

    thread A completes its unlock_file() and _fp_close() calls

    thread B completes its open() call

    thread A then barfs on its unlink() call (because you can't delete
    an open file on Windows, and thread B has the file open now)

Is that right?  If so, that's not the only way it can fail.  For example,

    before thread A does unlock_file(), thread B completes open() and
    tries to do lock_file()

    thread B will barf then (Windows file locks aren't "advisory" -- you
    can't lock a file that's already locked on Windows)

Example:

>>> import ZODB
>>> from ZODB import lock_file
>>> f = open('whatever', 'w')
>>> lock_file.lock_file(f)
>>> g = open('whatever', 'r')
>>> lock_file.lock_file(g)  # attempt to lock the same physical file twice
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "ZODB\lock_file.py", line 33, in lock_file
    _LockFile(file.fileno(), 0, 0, 1, 0)
winlock.error: 33
>>>

> Therefore, if another thread already uses the lock-file, the first thread
> is unable to delete it and the 'Permission denied' exception is thrown.

If I'm understanding you, that's one failure mode, but not the only failure
mode.

> As a workaround we put the os.unlink - statement in a try-except-block.
>
> def close(self):
>     if self._fp is not None:
>         unlock_file(self._fp)
>         self._fp.close()
>         try:
>             os.unlink(self._path)
>             self._fp = None
>         except:
>             pass

That can't stop the other failure mode above.
 
> How to correct the error best?

Trying to open a file in one thread, while simultaneously closing it another
thread, is the deeper problem here, and seems inherently ill-defined.  Do
you really need to do that?  Since yours is the first report of this, it
can't be a popular vice <wink>.