[Zope-dev] RE: [ZODB-Dev] [Warning] Zope/ZEO clients: subprocesses can lead tonon-deterministic message loss:MAYBE NOT

[sathya]

hello tim,
thanks for the clarification below and also the pointers to the posix 
behaviour of fork. The Warning about Zope/ZEO clients in the subject 
line certainly caused some alarm bells to go off.

I am assuming now that dieters description below of using forks does not 
gel with the ZOPE/ZEO process model i.e  there are no forks being called 
within the code to cause the asyncore thread to be cloned (  even if one 
were using a non posix compliant thread lib like native solaris it would 
still be a non issue ).

At least thats how I see it so far !

please let me know if anybody  thinks otherwise :)

Regards
Sathya

> [sathya]
> 
>>so can we safely assume that zeo does not mix the asyncore implementation
>>with  forks or threads and hence does not suffer from the "child
>>concurrently operating on sockets along with parent" syndrome that
>>dieter is experiencing ? appreciate any clarifications.
> 
> 
> It's normal for a ZEO application to run asyncore in its own thread.  I
> don't really understand what Dieter is seeing, though:
> 
> [Dieter]
> 
>>  When a process forks the complete state, including file descriptors,
>>  threads and memory state is copied and the new process
>>  executes in this copied state.
>>  We now have 2 "asyncore" threads waiting for the same events.
> 
> 
> A problem is that it's *not* the case that a POSIX fork() clones all
> threads.  Only the thread calling fork() exists in the child process.
> There's a brief but clear discussion of that here:
> 
>     http://www.opengroup.org/onlinepubs/009695399/functions/fork.html
> 
> POSIX doesn't even have a way to *ask* that all threads be duplicated, for
> reasons explained there.
> 
> Last I heard, Dieter was running LinuxThreads, which fail to meet the POSIX
> thread spec in several respects.  But, AFAICT, fork() under LinuxThreads is
> the same as POSIX in this particular respect (since threads are distinct
> processes under LinuxThreads, it would be bizarre if a fork() cloned
> multiple processes!).  I believe native Solaris threads act as Dieter
> describes, though (fork() clones all native Solaris threads).
> 
> Dieter, can you clarify which OS(es) and thread package(s) you're using
> here?  Do the things you're doing that call fork() (directly or indirectly)
> actually run from the thread running asyncore.loop()?  That's the only way a
> POSIX fork() should end up with a clone of the thread running the asyncore
> loop.  But then the subsequent exec (if you're doing system() or popen())
> should wipe out the cloned asyncore code before the child process returns to
> asyncore.
>