Argl, why does this mailing list not use the reply-to: field? ---------- Forwarded message ---------- From: Patrick Gerken <do3ccqrv@googlemail.com> Date: Sat, Apr 25, 2009 at 15:43 Subject: Re: [Zope] Building a fast, scalable yet small Zope application To: "Morten W. Petersen" <morten@nidelven-it.no> On Sat, Apr 25, 2009 at 13:24, Morten W. Petersen <morten@nidelven-it.no> wrote:
Hi,
I'm considering building a large scale, but small in features site. It will contain lots of small objects (millions, tens of millions, hundreds of millions) of objects, where each object has a couple of strings and maybe some other light attributes.
So far, I've been contemplating disabling undo (if that's possible), and using BTree structures, maybe segmenting objects into different groups (folders) to further speed up lookups. Scalability is also an issue, should I consider using RelStorage? Should I consider using the ZCatalog for faster lookups?
Has anyone else developed something similar? Are there Zope product examples out there that fit the bill?
Hi, I once was involved in developing a system that supposedly should be able to handle a very big number of objects. They chose to put all the base objects into mysql too, and use mysql for object lookup. I do not know what type of benchmarks they did to conclude that this was the best choice. But they regulary did queries for which relational databases were made. If your data contains lots of small items, that are all similar of content, and you want to do queries that involves many of these attributes, you might want to take a look at orm tools. relstorage will not help you with your queries, because it stores everything as a pickle, afaik. I suggest you experiment a bit. Create 100 million objects, and do some of the actions you are planning to do on them. Best regards, Patrick