[Zope-dev] RE: [ZODB-Dev] [Warning] Zope/ZEO clients: subprocesses can lead tonon-deterministic message loss

[Tim Peters]

[sathya]
> so can we safely assume that zeo does not mix the asyncore implementation
> with  forks or threads and hence does not suffer from the "child
> concurrently operating on sockets along with parent" syndrome that
> dieter is experiencing ? appreciate any clarifications.

It's normal for a ZEO application to run asyncore in its own thread.  I
don't really understand what Dieter is seeing, though:

[Dieter]
>   When a process forks the complete state, including file descriptors,
>   threads and memory state is copied and the new process
>   executes in this copied state.
>   We now have 2 "asyncore" threads waiting for the same events.

A problem is that it's *not* the case that a POSIX fork() clones all
threads.  Only the thread calling fork() exists in the child process.
There's a brief but clear discussion of that here:

    http://www.opengroup.org/onlinepubs/009695399/functions/fork.html

POSIX doesn't even have a way to *ask* that all threads be duplicated, for
reasons explained there.

Last I heard, Dieter was running LinuxThreads, which fail to meet the POSIX
thread spec in several respects.  But, AFAICT, fork() under LinuxThreads is
the same as POSIX in this particular respect (since threads are distinct
processes under LinuxThreads, it would be bizarre if a fork() cloned
multiple processes!).  I believe native Solaris threads act as Dieter
describes, though (fork() clones all native Solaris threads).

Dieter, can you clarify which OS(es) and thread package(s) you're using
here?  Do the things you're doing that call fork() (directly or indirectly)
actually run from the thread running asyncore.loop()?  That's the only way a
POSIX fork() should end up with a clone of the thread running the asyncore
loop.  But then the subsequent exec (if you're doing system() or popen())
should wipe out the cloned asyncore code before the child process returns to
asyncore.