[ZODB-Dev] Twisted, ZODB and zc.twist

[Gary Poster]

On Jul 29, 2011, at 4:38 AM, Erik Allik wrote:

> Thanks for another clear answer! I'll simply ignore zc.twist (for now
> at least) and set up a classic thread pool.
> I'll still probably use zc.twist's source as an example of how to
> write transaction retrial on ConflicErrors and ClientDisconnecteds as
> I haven't found anything similar elsewhere.

Cool.  FWIW, http://pypi.python.org/pypi/zc.queue does something similar without threads.  I prefer it (despite the fact that I said that "this testing approach has known weaknesses") if your use cases allow it.

Gary

> 
> Erik
> 
> 
> 
> 2011/7/29 Gary Poster <gary.poster at gmail.com>:
>> 
>> On Jul 28, 2011, at 6:38 PM, Erik Allik wrote:
>> 
>>> Gary, since you're the author of zc.twist (as I understand), I take
>>> this as an exhaustive answer, and thank you :)
>>> 
>>> However, I'm still curious to know zc.twist would provide any benefit
>>> when used together with a thread pool.
>>> The thread examples in its README (which I find hard to understand)
>>> indicate that there must be some meaning to using zc.twist with
>>> threads.
>> 
>> The threads in the README are there to let me explore a running reactor within the doctest.  They are not anything to emulate.
>> 
>> The README overstates zc.twist's usefulness.  As you point out, it blocks.  If you assume that the ZODB is functioning normally, it won't block "much" but with standard definitions of blocking, it fails.  For zc.async, that's fine--but a "while True" loop and a time.sleep() to give some breathing room probably would have been fine, with hindsight, and much simpler.  The use of Twisted is a very questionable convenience, and the way zc.async uses Twisted is not to be copied 99% or more of the time.
>> 
>> Gary
>> 
>> 
>>> 
>>> Thanks again and best regards,
>>> Erik
>>> 
>>> 
>>> 
>>> 2011/7/29 Gary Poster <gary.poster at gmail.com>:
>>>> Hello Erik.  As the author, I'll suggest that you move on to a ZODB connection pool used by the usual Twisted-provided thread pool.  zc.twist is maybe interesting and maybe clever, but of very limited usefulness.  I suspect it is only used in zc.async, and that's probably as it should be.  It's a case of premature generalization.
>>>> 
>>>> If you are going to use Twisted in anything like a normal way, and the ZODB in anything like a normal way, use a connection pool and threads.
>>>> 
>>>> Best,
>>>> 
>>>> Gary
>>>> 
>>>> 
>>>> On Jul 28, 2011, at 4:12 PM, Erik Allik wrote:
>>>> 
>>>>> Hello everyone,
>>>>> 
>>>>> I hope this is the right mailing list to post this question to.
>>>>> 
>>>>> I'm trying to build an Twisted application that uses ZODB and I've
>>>>> also found zc.twist.
>>>>> I've read its README as well as the source code to try to understand
>>>>> how exactly and why I should use zc.twist, and also asked the same
>>>>> questions on irc.freenode.net #zope but without a conclusive answer.
>>>>> 
>>>>> What I'm not able to understand is whether using zc.twist will allow
>>>>> me to do all ZODB access in the main Twisted thread, or if still need
>>>>> to set up a thread pool and deferToThread any code that accesses the
>>>>> DB.
>>>>> 
>>>>> The README of zc.twist has:
>>>>> "Everything can be done within the main thread, so it can be full-bore
>>>>> Twisted usage, without threads."
>>>>> 
>>>>> However, looking at the source code of zc.twist, I cannot find any
>>>>> reference to async communication (callbacks/polling) with ZODB, so
>>>>> it's difficult to conclude that all DB access is non-blocking and can
>>>>> be done in a single thread. Also, there are some examples with threads
>>>>> in the README.
>>>>> 
>>>>> I would highly appreciate if somebody with a clearer understanding of
>>>>> this could explain to me whether I in fact need to set up a thread
>>>>> pool, and in any case, what exactly is zc.twist for regardless of
>>>>> whether it needs to be used with a threadpool or not.
>>>>> 
>>>>> Thanks in advance!
>>>>> 
>>>>> Regards,
>>>>> Erik Allik
>>>>> 
>>>>> P.S. I'd be very thankful for any references to open source
>>>>> projects/code using ZODB with Twisted, or even zc.twist (besides
>>>>> zc.async).
>>>>> _______________________________________________
>>>>> For more information about ZODB, see the ZODB Wiki:
>>>>> http://www.zope.org/Wikis/ZODB/
>>>>> 
>>>>> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
>>>>> https://mail.zope.org/mailman/listinfo/zodb-dev
>>>> 
>>>> 
>> 
>>