Once upon a time I wrote an IRCLog product for handhelds.org, using the a2k (accutron 2000) ircbot (python based) to relay the irc messages into zope. The product was written for Zope 2.2 (IIRC) and while still setup, the accutron bot is no longer recording IRC messages. In any case... I made a design decision that was pretty bad when designing the original IRCLog product. That decision was to store all of the irc messages in arrays, with each message adding to the array and then storing it in the ZODB. This caused the ZODB to grow very fast (considering 500-1000+ irc messages a day) as each copy of the array was stored in the ZODB. So. Recently I decided to port the zope product to 2.4.1 and update it to use a more resonable storage mechanism for the messages. Here are the following storage mechanisms I've come up with: 1) Create another class and have the main zope product create one of these classes for each message and store that (in a folderish way) in the ZODB. This should cut down on the rapid growth of the ZODB (vs. precvious versions) and keep everything in zope. 2) Write the irc messages to a text file and have the IRCLog parse/display that. I'm not too fond of this because it really doesn't take advantage of zope's abilities, the ZODB, etc. 3) Write the irc messages to an SQL database (we run MySQL on hh.org; but I could run postgres as well). I don't have much experience with Zope+SQL databases and I'm not too eager to learn right now...But if it offers some advantages I may do it this way (Note: SQL doesn't scare me.). But this also has the disadvantage that I'm again going outside of Zope. So ... here are my questions: 1) Which one of the above (or other) storage mechanisms would you use and why? 2) What do I need to pay attention to porting from zope 2.2 to zope 2.4.1? Note: I've been away from zope development (and python) for a while (doing java...which I've just decided to totally hate .. but that's another story) .. So If I don't get something ... please bear with me.... :-) -- Edward Muller - edwardam@home.com - edwardam@handhelds.org There are two kinds of people, those who do the work and those who take the credit. Try to be in the first group; there is less competiton there.
Edward Muller writes:
.... considering 500-1000+ irc messages a day .... Here are the following storage mechanisms I've come up with: 1) Create another class and have the main zope product create one of these classes for each message and store that (in a folderish way) in the ZODB. This should cut down on the rapid growth of the ZODB (vs. precvious versions) and keep everything in zope.
2) Write the irc messages to a text file and have the IRCLog parse/display that. I'm not too fond of this because it really doesn't take advantage of zope's abilities, the ZODB, etc.
3) Write the irc messages to an SQL database (we run MySQL on hh.org; but I could run postgres as well). I don't have much experience with Zope+SQL databases and I'm not too eager to learn right now...But if it offers some advantages I may do it this way (Note: SQL doesn't scare me.). But this also has the disadvantage that I'm again going outside of Zope.
So ... here are my questions: 1) Which one of the above (or other) storage mechanisms would you use and why? Similar questions have been discussed in the past. Search the mailing list archive!
Personnally, I do not trust ZCatalog too much. I would go for it when I have a small to medium number of objects to catalog: say several thousand. I would go for a RDB solution when I had in the order of a million or more objects...
2) What do I need to pay attention to porting from zope 2.2 to zope 2.4.1? Should not be too difficult.
Just do it and look at the few tracebacks you will get... Dieter
participants (2)
-
Dieter Maurer -
Edward Muller