- Zope-Dev - Zope lists

More resilient indexes
by Roché Compaan 19 Sep '07

19 Sep '07

Hi there Are the current indexing solutions that we have in Zope 2 and 3 the best that we can have? Is there any way to improve the amount of concurrent indexing a field or text index can handle. The Zope 3 implementation doesn't look significantly different to the Zope 2 one in that it still uses a BTree for forward and reverse index. I have a basic stress test where 10 concurrent threads index 10 random words in a FieldIndex and get too many conflict errors to even consider this a usable scalable indexing solution. I don't really want to index objects in another backend. I really would like to make the ZODB work for me here. Are there solutions here? Can one employ some of the QueueCatalog conflict resolution strategies to make indexes more resilient? Or should one use some locking strategy instead? -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za

4 6

security problem in an monkey-patch
by Joachim Schmitz 19 Sep '07

19 Sep '07

Hi, I have monkey-patched the QueueCatalog to adopt it to our needs, which works fine. I now wanted to introduce a new feature: The QueueCatalog should be bypassed during mass-import of data. So I introduced a new variable "_bypass", and new getBypassQueue() and setBypassQueue methods in the monkey-patch: security.declareProtected(view_management_screens, 'getBypassQueue') def getBypassQueue(self): "get _by_pass" if not hasattr(self,"_bypass"): self._bypass = False return self._bypass security.declareProtected(view_management_screens, 'setBypassQueue') def setBypassQueue(self, bypass=False): "set _bypass" self._bypass = bypass from Products.QueueCatalog.QueueCatalog import QueueCatalog QueueCatalog.getBypassQueue = getBypassQueue QueueCatalog.setBypassQueue = setBypassQueue I can invoke these methods from the url like: ../portal_catalog/setBypassQueue?bypass=1 and ../portal_catalog/getBypassQueue displays a 1 But when I do a: <input type="checkbox" name="enable_bypass" tal:attributes="checked here/portal_catalog/getBypassQueue" /> I get: Unauthorized: The container has no security assertions. Access to 'getBypassQueue' of (QueueCatalog at /uniben/portal_catalog) denied. What I am missing here. -- Gruß Joachim

3 3

Zope Tests: 5 OK
by Zope Tests Summarizer 19 Sep '07

19 Sep '07

Summary of messages to the zope-tests list. Period Tue Sep 18 12:00:00 2007 UTC to Wed Sep 19 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Sep 18 20:50:14 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008361.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Sep 18 20:51:50 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008362.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Sep 18 20:53:21 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008363.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Sep 18 20:54:51 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008364.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Sep 18 20:56:21 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008365.html

1 0

Re: Fwd: [Zope-dev] Options replacing DateTime with datetime!?
by Lennart Regebro 18 Sep '07

18 Sep '07

On 9/17/07, Gustavo Niemeyer <gustavo(a)niemeyer.net> wrote: > > 2. "Timezones in the form of EST5EDT or US/Eastern doesn't work. > > > > Both these can be avoided by keeping the old time and timezone > > interpretation around. > > I'm not aware about that. In fact, I'm curious about what the problem > is. EST5EDT may be seen in the test suite several times, using different > sources of information even. > > >>> tz.gettz("EST5EDT") > tzfile('/usr/share/zoneinfo/EST5EDT') > > >>> tz.tzstr("EST5EDT") > tzstr('EST5EDT') OK, I'm just going after my old notes here, and they could be wrong. It could be that these types of timezones doesn't work in a datetime string? > > The third one is the the modules concept of local timezone is set when > > the module is imported, and there was no way to change it afterwards, > > dateutil can get timezones from several different sources and formats. > tzlocal is based on what libc exposes, and I don't advice it to be used > if any better source of information is available. If the fact that > some of its data is initialized as module import time is bothering you, > I can definitely change it, but please keep in mind that there are other > gotchas (variables in the time module won't change unless you call tzset, > for instance) The important part is that there is some sort of way to tell the module what the local timezone is, so that you can test conversions. > I recomment using tz.gettz() instead, which doesn't suffer from any of > the problems you described, and provides much better information > about the local timezone. Doens't help as long as conversions use a concept of a local timezone which isn't controllable. > Making testing impossible is a bit overstated, IMO. I use TDD in a > daily basis, and testing that environment seems far from challenging. > Time-related logic is tricky to test sometimes, of course, but saying > that dateutil is useless because tzlocal is initialized at import time > is funny, at least. Of course nothing is impossible in computing. But it makes it impossible without actually modifying the modules code somehow, and in my book, it is reasonable to call that impossible. If I monkey-patch the module for testing, then am I really testing what goes on in production? -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

2 1

Zope Tests: 5 OK
by Zope Tests Summarizer 18 Sep '07

18 Sep '07

Summary of messages to the zope-tests list. Period Mon Sep 17 12:00:00 2007 UTC to Tue Sep 18 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Mon Sep 17 20:53:49 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008356.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Mon Sep 17 20:55:39 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008357.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Mon Sep 17 20:57:09 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008358.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Mon Sep 17 20:58:43 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008359.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Mon Sep 17 21:00:18 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008360.html

1 0

Re: Fwd: [Zope-dev] Options replacing DateTime with datetime!?
by Sidnei da Silva 17 Sep '07

17 Sep '07

(Sending reply to the list, since Gustavo doesn't seem to be subscribed.) On 9/17/07, Gustavo Niemeyer <gustavo(a)niemeyer.net> wrote: > > Hey Lennart, > > > 1. Giving the timezone string 'GMT+3' meant that you got the timezone > > GMT-3 back. This was done intentionally and I had some discussion with > > Gustavo about that. > > True.. after these discussions I'm sure that the current implementation > isn't ideal. I'll try to fix that in the next dateutil release. > > > > 2. "Timezones in the form of EST5EDT or US/Eastern doesn't work. > > > > Both these can be avoided by keeping the old time and timezone > > interpretation around. > > I'm not aware about that. In fact, I'm curious about what the problem > is. EST5EDT may be seen in the test suite several times, using different > sources of information even. > > >>> tz.gettz("EST5EDT") > tzfile('/usr/share/zoneinfo/EST5EDT') > > >>> tz.tzstr("EST5EDT") > tzstr('EST5EDT') > > > > The third one is the the modules concept of local timezone is set when > > the module is imported, and there was no way to change it afterwards, > > dateutil can get timezones from several different sources and formats. > tzlocal is based on what libc exposes, and I don't advice it to be used > if any better source of information is available. If the fact that > some of its data is initialized as module import time is bothering you, > I can definitely change it, but please keep in mind that there are other > gotchas (variables in the time module won't change unless you call tzset, > for instance) > > I recomment using tz.gettz() instead, which doesn't suffer from any of > the problems you described, and provides much better information > about the local timezone. > > > > in effect making any testing impossible, as any tests in converting > > timezones will only work in the timezone of the one who wrote the > > tests. That's a huge problem when you need to test that timezones work > > properly, and IMO opinion makes the module pretty much useless for any > > Making testing impossible is a bit overstated, IMO. I use TDD in a > daily basis, and testing that environment seems far from challenging. > Time-related logic is tricky to test sometimes, of course, but saying > that dateutil is useless because tzlocal is initialized at import time > is funny, at least. > > Here is a hot-fix for tzlocal, if that's the problem you're blocked on. > > class tzlocal(dateutil.tz.tzlocal): > > @property > def _std_offset(self): > return datetime.timedelta(seconds=-time.timezone) > > @property > def _dst_offset(self): > if time.daylight: > return datetime.timedelta(seconds=-time.altzone) > else: > return self._std_offset > > dateutil.tz.tzlocal = tzlocal > > > -- > Gustavo Niemeyer > http://niemeyer.net > -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

1 0

Zope Tests: 5 OK
by Zope Tests Summarizer 17 Sep '07

17 Sep '07

Summary of messages to the zope-tests list. Period Sun Sep 16 12:00:00 2007 UTC to Mon Sep 17 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sun Sep 16 20:51:10 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008351.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sun Sep 16 20:52:41 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008352.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Sep 16 20:54:12 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008353.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Sep 16 20:55:42 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008354.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Sep 16 20:57:17 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008355.html

1 0

request.locale
by Laurence Rowe 16 Sep '07

16 Sep '07

Quite a lot of zope3 code (zc.datetimewidget for instance) expects to be able to access request.locale. ZPublisher does not provide this and to get around the limitation you must manually set request.locale in your view using Products.CMFDefault.formlib.form.getLocale. This seems brittle. I think it would be desirable to include such functionality in Zope 2. If there is support for this, where should it go? ZPublisher.HTTPRequst or somewhere in Five seem like prime candidates. Laurence

5 9

Zope Tests: 5 OK
by Zope Tests Summarizer 16 Sep '07

16 Sep '07

Summary of messages to the zope-tests list. Period Sat Sep 15 12:00:00 2007 UTC to Sun Sep 16 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sat Sep 15 20:48:59 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008346.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sat Sep 15 20:50:33 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008347.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sat Sep 15 20:52:03 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008348.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sat Sep 15 20:53:33 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008349.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Sat Sep 15 20:55:03 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-September/008350.html

1 0

New Contributor References
by Jim Fulton 16 Sep '07

16 Sep '07

Lately, I've started being a bit more careful about accepting new contributors. In particular, I'm requesting that new contributors provide the name of an existing contributor who is familiar with them and their work. I (or someone) really need to add this to the contributor agreement, but I'm not sure when I'll have time. Now, It's taking a fair bit of time writing new contributors to ask for references. I encourage folks to request committer access, I just ask that people provide the names of one or more existing committers who can vouch for them. Thanks. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

2 1