[Zope-dev] Re: What catalog/index to use ...

Joachim Werner joe@iuveno-net.de
Fri, 8 Nov 2002 19:49:24 +0100


Hi!

Some additional remarks: While making the splitting dependent on the locale
settings (as done in the old TextIndex) helps with most use cases, I'm not
sure if that is the right thing to do in the long run. Locale settings are
good for client software, i.e. if you want to have a program behave German
for a German user etc.

But a web server might be located in the U.S., but frequented by
German-speaking and Spanish-speaking users. Or even users from China or
Japan. In these cases only Unicode will help I think. After all you can not
have more than one locale at a time. But honestly I still don't understand
the Unicode thing good enough. My main concern is whether a Unicode-enabled
site will still work with older browsers and for all platforms ...

> Casey Duncan wrote:
> > The main reason I have not merged this already is that I lack a sample
to make
> > a new test with. If someone can provide me with some content samples
that
> > break now, but work with the patch, I will make a new test and checkin
the
> > fix for 2.7 perhaps 2.6.1 if desired.
> >
> > -Casey
>
> hi Casey,
>
> try some words with "german umlaute". things like:
>
> mülltonne
> waschbär
> behörde
> überflieger
>
> the last one will work without the patch. explanation: the first
> character is splitted away [non-ascii-character] [both for storing the
> word in the Lexicon and resolving the query-words through the
> queryparser]. so it will in both cases end up in
>
> berflieger
>
> searching for 'überflieger' will give you correct results. this is the
> reason, why some people think, that ZCTextIndex works with german
> 'umlaute', but it does not...;-)

This patch will probably not hurt anybody. And it would make ZCTextIndex
behave like TextIndex.

OT:

I don't want to be too pedantic about that, but usually I'd expect a
replacement to really replace all of the functionality of the thing it
replaces.
TextIndex was locale-aware (and this was even documented somewhere), so
switching to ZCTextIndex should not break anything, at least not in a Zope
final.

But that's what I've told you all the time: Why do you make things final
releases before they are really tested? 2.6.0 has a really bad bug with the
DateTime module (Lennart Regebro has provided a fix for it:
http://www.zope.org/Members/regebro/datetime_260_fix) that was introduced
after 2.6.0b1. This just shouldn't be possible ...

Anybody listening? ;-)

Cheers

Joachim