--On Freitag, 8. November 2002 19:49 +0100 Joachim Werner <joe@iuveno-net.de> wrote:
Hi!
Some additional remarks: While making the splitting dependent on the locale settings (as done in the old TextIndex) helps with most use cases, I'm not sure if that is the right thing to do in the long run. Locale settings are good for client software, i.e. if you want to have a program behave German for a German user etc.
But a web server might be located in the U.S., but frequented by German-speaking and Spanish-speaking users. Or even users from China or Japan. In these cases only Unicode will help I think. After all you can not have more than one locale at a time. But honestly I still don't understand the Unicode thing good enough. My main concern is whether a Unicode-enabled site will still work with older browsers and for all platforms ...
Please note that former Zope versions already include a dedicated unicode-aware splitter that is already usable with the old TextIndex and maybe with ZCTextIndex. TextIndexNG resolves all these issues by doing the complete internal processing by converting the data into unicode. Every single processing step only handles unicode data. Most older browsers should be able to handle at least UTF-8 as character set. This is sufficient for most cases. =aj