Nestor A. Diaz L. writes:
I have to make a system to store news, so i have to migrate a old system using perl cgi's and perform a search on text files to use Zope as the search engine however i don't know if i have to use a SQL database like MySQL or use the ZCatalog, i need full text search, and ZCatalog fits on the true decision however the current database news is about 350M and i would like to extend it to store images and file, so is possible that at the final year if everyting comes ok it outperforms the 1GB data If you have a database or search engine that handles full text, I would go for it. However, most relational databases have difficulties with full text. I doubt, that MySQL can handle it well.
The ZCatalog as it is now, is nice for small to medium size content. I, personally, would not use it for content bigger than 50 to 100 MB. Due to some weeknesses in the BTree implementation (which is used by ZCatalog to maintain its indexes) a considerable part of the index structure is rewritten if the index changes. This may lead to a fast growing ZODB with the need for frequent "pack"'s. Before you decide for ZODB, you should make some tests: something like this: * index about 10 MB of content * change content * search with the catalog * index another 10 MB of content (you now have 20 MB content) * change content * search with the catalog For each step: Look how the size of the ZODB changes: before and after packing Look how the main memory requirements changes (especially during indexing) Look how the timings changes (for all relevant operations: indexing, packing, searching) Try to extrapolate your results. You may need to add more 10 MB content chunks to recognize a tendancy. Dieter