[Zope] future searching of chinese text... (was: that JPython thing)

[chas]

>> 3. support of unicode
>
>Ah, as soon as python does, we do.  Also, soon there will be a Japanese
>Vocabulary to support Japanese searching of text, and after that we are
>going to try Chinese.  In Zope 2.2, using these examples, you can create
>a Vocabulary object for the language of your preference.

Just on this subject :

a) You may find the Perl mandarin text-splitter at www.mandarintools.com
   very useful. I rewrote it in Python once but my Perl sucks so you may 
   wish to do this yourself. Otherwise, e-mail me for copy ... it's so 
   short I'm sure you'd rather not trust mine though.

b) What algorithm do you use in your searching of text ? Is it just a
   simple frequency tally ? 

chas