[ANN] TextIndexNG 2.0 alpha 1 released
I am pleased to announce the release of TextIndexNG 2.0 alpha1 . What's new in TextIndexNG 2.0? - Relevance ranking of search results added. Searches are now ranked using an extended cosine measure. The cosine measure is based on a vector model and calculates the document "score" based on the frequency of the query terms inside the document result set. - Much faster phrase/near search: the old implementation of TextIndexNG had to perform a very expensive job at query time when phrase/near search was performed. Re-using the !WidCode module of !ZCTextIndex made this operation less expensive. - Left-truncation added: TextIndexNG can be configured creation-time time to support left-truncation (means you can search for "*suffix") - optional auto-expansion support: This optional feature also to get better search results when some of the query terms could not be found. The index expands a query term "foo" to "foo*" if there was no hit for "foo". This expansion is currently global for the index. This feature will be available on a per-query basis in a later version. (Auto-expansion will be extended in a later version to search for similiar terms) - improved HTML converter: now using Chris Withers "Strip-o-Gram" module instead of the Strip-Tag-Parser - added converter for text/sgml - Similarity search (soundex, metaphone, doublemetaphone) dropped and replace with a more general approach and language indepedant approach using the Levenshtein distance. - internal code cleanup, more unittests Not implemented yet - improved support for multilingual documents - range searches like "Fi..Foo" - substring searches "*substring*" - optional improved ranking for terms based on their relative positions inside a document Installation notes TextIndexNG 2.0 is *not compatible* with TextIndexNG 1.0 so there is currently no migration path for existing indexes. - **BACKUP YOUR Data.fs first!!!** - remove any existing TextIndexNG from your ZCatalog indexes - shutdown Zope - remove the old Products/TextIndexNG directory - untar the tarball in the Products folder - recompile and install extension modules (see InstallationInstructions) - restart Zope - re-create the indexes and re-index Download: http://sourceforge.net/project/showfiles.php?group_id=50052 Project Wiki: http://www.zope.org/Members/ajung/TextIndexNG/
Andreas Jung wrote:
- Relevance ranking of search results added. Searches are now ranked using an extended cosine measure. The cosine measure is based on a vector model and calculates the document "score" based on the frequency of the query terms inside the document result set.
Can you show an example of how this would be used in a ZCatalog search involving some other indexes where the end result of the search should be sorted by the relevence rank of the TextIndexNG? cheers, Chris
--On Mittwoch, 7. Mai 2003 9:39 Uhr +0100 Chris Withers <chrisw@nipltd.com> wrote:
Andreas Jung wrote:
- Relevance ranking of search results added. Searches are now ranked using an extended cosine measure. The cosine measure is based on a vector model and calculates the document "score" based on the frequency of the query terms inside the document result set.
Can you show an example of how this would be used in a ZCatalog search involving some other indexes where the end result of the search should be sorted by the relevence rank of the TextIndexNG?
I can't :-) The reason is the same as with ZCTextIndex :-) Andreas
participants (2)
-
Andreas Jung -
Chris Withers