I'm in the process of trying to build a reliable, accurate search engine using a ZCatalog. In order to keep the catalog contents up to date, I added an Xron method to reindex the catalog every hour. This worked fine on a small site, but for a site with a catalog of 1300 documents the data.fs file grows from a few hundred meg to several gig before long, and apache/zope dies. Another problem is that Xron seems to trigger its methods as many times as it 'missed' if the server is stopped for a while. I'm only indexing dtml documents, and the ZCatalog doesn't use any metadata or special indexes. Are there any ways to prevent the catalog updates being stored in the ZODB, so no undo history is saved for the ZCatalog? Thanks -- Alex This e-mail and any attached files are confidential and are intended solely for the use of the individual or entity to whom they are addressed. This communication represents the originators personal views and opinions, which do not necessarily reflect those of The Royal College of Surgeons of England. If you are not the intended recipient, or the person responsible for delivering the e-mail to the intended recipient, be advised that you have received this e-mail in error, and that any use, dissemination, forwarding, printing or copying of this e-mail is strictly prohibited. http://www.rcseng.ac.uk Registered Charity Number: 212808
Luton, Alex wrote:
I'm in the process of trying to build a reliable, accurate search engine using a ZCatalog. In order to keep the catalog contents up to date, I added an Xron method to reindex the catalog every hour. This worked fine on a small site, but for a site with a catalog of 1300 documents the data.fs file grows from a few hundred meg to several gig before long, and apache/zope dies. Another problem is that Xron seems to trigger its methods as many times as it 'missed' if the server is stopped for a while. I'm only indexing dtml documents, and the ZCatalog doesn't use any metadata or special indexes. Are there any ways to prevent the catalog updates being stored in the ZODB, so no undo history is saved for the ZCatalog?
Hi, you should use: http://www.zope.org/Members/Kaivo/DTMLDocumentExt to make your Documents CatalogAware... Then you don't have to reindex all your documents by Xron... each document will reindex itself when it is changed. Please note: There's a bug with keyboard-args in DTMLDocumentExt with manage_changeProperties... please change the according lines in __init.py__ to: 138: def manage_changeProperties(self, REQUEST=None, **kw): 139: r = original_changeProperties(self, REQUEST, **kw) Cheers, Maik
participants (2)
-
Luton, Alex -
Maik Jablonski