[Zope-Checkins] CVS: Zope/lib/python/Products/PluginIndexes/TextIndexNG/src/PyStemmer/test - test.py:1.1.2.1

[Andreas Jung]

Update of /cvs-repository/Zope/lib/python/Products/PluginIndexes/TextIndexNG/src/PyStemmer/test
In directory cvs.zope.org:/tmp/cvs-serv30556/PyStemmer/test

Added Files:
      Tag: ajung-textindexng-branch
	test.py 
Log Message:
added PyStemmer


=== Added File Zope/lib/python/Products/PluginIndexes/TextIndexNG/src/PyStemmer/test/test.py ===
#!/usr/bin/env python2.1

import os
import Stemmer
import time

for lang in Stemmer.availableStemmers():
    print 'checking',lang

    S = Stemmer.Stemmer(lang)
    SF = S.stem

    try:
        voc = open(os.path.join('..',lang,'voc.txt')).readlines()
    except:
        voc = open(os.path.join(lang,'voc.txt')).readlines()

    try:
        out = open(os.path.join('..',lang,'output.txt')).readlines()
    except:
        out = open(os.path.join(lang,'output.txt')).readlines()

    ts = time.time()

    for i in range(len(voc)):
        v = voc[i][:-1]
        assert SF(v)==out[i][:-1]

    print time.time() - ts,'seconds'


    voc = [ x.strip() for x in voc]

    ts = time.time()
    res = SF(voc)
    assert len(res)==len(voc)

    print time.time() - ts,'seconds'