PossitionIndex (was: Re: [Zope-dev] ZCatalog phrase indexingrevisited)

[Rik Hoekstra]

Chris McDonough wrote:
> 
> It just occurred to me that depending on the splitter to do
> positions makes it impossible to alter the splitter without
> reindexing the whole text index... but I think this is a
> reasonable tradeoff.  Other opinions welcome.
> 

This raises the question how dependent the splitter on the paticularities of the
document source - I do not really see how different splitters could be useful
for one single document. This is perhaps less obvious than it appears, as you
may want to use different splitters for documents in different languages. Taken
as a whole I would say choosing a splitter would be a decision that had to be
taken at indexing time anyway. But perhaps it's just my imagination that is
lacking. 

There is a much greater dependence on the lexicon here. And indeed several
different lexicons could be applied to a set of documents depending of what is
wanted. 

my 2 cents

Rik