The largest installation with TextIndexNG running I have seen was about 70,000 documents large. It ran on a fast Athlon machine with 256 MB. I assume you are running into some kind of memory problem. I suggest to run "top" when you are trying to index the documents. Watch the memory usage of the running Zope process and checkout your swap process. Also you can try to play around with the number of subtransactions uses. I have seen a memory explosion on a solaris system some time ago but never seen such a memory problem under Linux i386. If you have further problems please contact me again. Andreas --On Sonntag, 8. Dezember 2002 17:26 -0600 David Mackay <mackay_d@bellsouth.net> wrote:
I am having a problem creating an index for about 276,000 documents. I successfully created a TextIndexNG index for the documents by indexing 50,000 at a time. If I tried to do everything at one time, it would work for about six or seven hours, and then return with zer0 items indexed. Before I added the indexed items, my database size was about 360MB. Well, I decided that I needed to make a few changes, so I cleared the index. Then I packed the database. However, the database size is now 2023.7M. When I try to repack the database, I get an error message telling me that there have been no changes since the last pack. Also, I can't add any indices. I get a message that the connection was refused when attempting to contact the server. There are no messages in Z2.log indicating that there is a problem, and no tracebacks from python. The environment is: Zope 2.5.1 Python 2.1.3 Linux (Red Hat 8.0) kernel 2.4.18-3 The machine has 768MB of memory and 11GB of free hard drive space.
Can anyone give me some pointers?
Dave
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
--------------------------------------------------------------------- - Andreas Jung http://www.andreas-jung.com - - EMail: andreas at andreas-jung.com - - "Life is too short to (re)write parsers" - ---------------------------------------------------------------------