Thanks for your response, Andreas, and thanks for writing TXNG in the first place. I really appreciate the contribution.
I recently installed TextIndexNG2 2.1.1
which is *pretty old*. Take a look at the v 2.2.0 which has been optimized over the time in different ways. Consider using StupidStorage as documented in the release notes.
Question re v 2.2.0: on this page http://www.zope.org/Members/ajung/TextIndexNG there are three possibilities 2.2.0, 2.2.0b1, 2.2.0b2. The last modified column indicates that 2.2.0 has been modified most recently, but I'm assuming that the b1 & b2 versions are in fact more recent. Is that true? Also, 2.2.0b1 is around 2MB, whereas the other two are 600K and I noticed after downloading 2.2.0 that stemming support seems absent. So, from all this I'm assuming that if I want stemming support I should grab the 2MB 2.2.0b1. Is that right? I'll look into StupidStorage.
I've been struck that if the number of search hits is high, TextIndexNG2 is much slower than ZCTextIndex. For example, if I do a search on 'podcast' (our site deals w/ podcasting) I get about 14,000 hits. ZCTextIndex returns the results in about 0.1 seconds; TextIndexNG2 takes 31 seconds or 300 times longer. In general, the more hits there are, the bigger the difference between the two search indexes.
Query speed depends on different things: the query, the implementation, the operations needed to be performed during the query. Because of some functionality TXNG needs to store much more information than ZCTextIndex. It did this as said above sometimes in a not so efficient way (see above). You might also look at TextIndexNG V3.
V3 is appealing but we're running 2.7.6 and I'm not sure that I want the hassle of installing Five at this point. When we go to 2.8, which we'll probably due in a couple of months, V3 will be an option. Thanks for your help, Francis