Fwd: Building a fast, scalable yet small Zope application
Argl, why does this mailing list not use the reply-to: field? ---------- Forwarded message ---------- From: Patrick Gerken <do3ccqrv@googlemail.com> Date: Sat, Apr 25, 2009 at 15:43 Subject: Re: [Zope] Building a fast, scalable yet small Zope application To: "Morten W. Petersen" <morten@nidelven-it.no> On Sat, Apr 25, 2009 at 13:24, Morten W. Petersen <morten@nidelven-it.no> wrote:
Hi,
I'm considering building a large scale, but small in features site. It will contain lots of small objects (millions, tens of millions, hundreds of millions) of objects, where each object has a couple of strings and maybe some other light attributes.
So far, I've been contemplating disabling undo (if that's possible), and using BTree structures, maybe segmenting objects into different groups (folders) to further speed up lookups. Scalability is also an issue, should I consider using RelStorage? Should I consider using the ZCatalog for faster lookups?
Has anyone else developed something similar? Are there Zope product examples out there that fit the bill?
Hi, I once was involved in developing a system that supposedly should be able to handle a very big number of objects. They chose to put all the base objects into mysql too, and use mysql for object lookup. I do not know what type of benchmarks they did to conclude that this was the best choice. But they regulary did queries for which relational databases were made. If your data contains lots of small items, that are all similar of content, and you want to do queries that involves many of these attributes, you might want to take a look at orm tools. relstorage will not help you with your queries, because it stores everything as a pickle, afaik. I suggest you experiment a bit. Create 100 million objects, and do some of the actions you are planning to do on them. Best regards, Patrick
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Patrick Gerken wrote:
Argl, why does this mailing list not use the reply-to: field?
That would be a "religious" question, and one which is not helpful to debate here.. E.g., see: http://www.unicom.com/pw/reply-to-harmful.html The Zope mailing lists are *all* organized that way, because the admins / long-time users (including myself) are adherents of the "munging considered harmful" denomination. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ9J3E+gerLs4ltQ4RAnIyAKDAsEHOn49A4rY7S5GZ84NCbypytACfdNGr yXP1QjIwUofepjFrwccbwjU= =eapg -----END PGP SIGNATURE-----
jajajajajajajajajajajajaja Amazing! I don't know how the other list doesn't collapse using reply-to field jajajajajajajajajajajajaja What wonders me more is how belligerent some in this list are with this question You cause the problem, at least try to don't threaten us with ban us if we send you private messages by mistake This kind of things are to don't take you seriously, sorry, but this cause a very bad impression of the people of this list 2009/4/26 Tres Seaver <tseaver@palladion.com>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Patrick Gerken wrote:
Argl, why does this mailing list not use the reply-to: field?
That would be a "religious" question, and one which is not helpful to debate here.. E.g., see:
http://www.unicom.com/pw/reply-to-harmful.html
The Zope mailing lists are *all* organized that way, because the admins / long-time users (including myself) are adherents of the "munging considered harmful" denomination.
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFJ9J3E+gerLs4ltQ4RAnIyAKDAsEHOn49A4rY7S5GZ84NCbypytACfdNGr yXP1QjIwUofepjFrwccbwjU= =eapg -----END PGP SIGNATURE-----
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
-- Mis Cosas http://blogs.sistes.net/Garito Zope Smart Manager http://blogs.sistes.net/Garito/670
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Garito wrote:
jajajajajajajajajajajajaja
Amazing!
I don't know how the other list doesn't collapse using reply-to field jajajajajajajajajajajajaja
I said "religious" because I meant that people have strong opinions about which style they prefer, and are not going to change them because others argue for the other style. I didn't say (or imply) that those who disagree with me are wrong / evil / stupid. I *did* say that this list is run according to the "no-munging" rules, and that it was a deliberate choice, not subject to debate.
What wonders me more is how belligerent some in this list are with this question
I don't see how my reply can be considered belligerent. Patrick asked "why doesn't the list munge 'Reply-To'?", and I replied with a link to an explanation, along with a statement that arguing the question would not be fruitful.
You cause the problem,
If you had actually read the link I sent, you would have learned about the problems caused by munging 'Reply-To'. I have participated in *lots* of mailing lists since the early nineties: those which do munge the header are more painful, and routinely have "oops, that was supposed to be a private reply" messages show up. The damage from such messages is *much* higher than the occasional "forget to CC the list" stuff on non-munging lists. Again, this is *opinion,* shared by lots of mailing list maangers / users, but certainly not by all.
at least try to don't threaten us with ban us if we send you private messages by mistake
I don't know what you are talking about here: I didn't threaten to ban anybody. I generally ask (politely) that people keep their replies on-list, and assume that the occasional off-list reply is an honest mistake. I *have* set up filters (in my own mail processing) dropping messages from a couple of users who persist in sending me private mail after being asked to keep the traffic on the list.
This kind of things are to don't take you seriously, sorry, but this cause a very bad impression of the people of this list
I'm sorry you feel that way. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ9cVx+gerLs4ltQ4RAoCdAJ9xOVCuNjNoxf5uMgTeaSQhiaXMNwCgpnvR GHTZGcy1s0hP1/HRJPxX5xE= =shhQ -----END PGP SIGNATURE-----
2009/4/27 Tres Seaver <tseaver@palladion.com>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Garito wrote:
jajajajajajajajajajajajaja
Amazing!
I don't know how the other list doesn't collapse using reply-to field jajajajajajajajajajajajaja
I said "religious" because I meant that people have strong opinions about which style they prefer, and are not going to change them because others argue for the other style. I didn't say (or imply) that those who disagree with me are wrong / evil / stupid. I *did* say that this list is run according to the "no-munging" rules, and that it was a deliberate choice, not subject to debate.
Making joke, man, that's the point about religious, isn't it?
What wonders me more is how belligerent some in this list are with this question
I don't see how my reply can be considered belligerent. Patrick asked "why doesn't the list munge 'Reply-To'?", and I replied with a link to an explanation, along with a statement that arguing the question would not be fruitful.
I mix other conversations with that
You cause the problem,
If you had actually read the link I sent, you would have learned about the problems caused by munging 'Reply-To'. I have participated in *lots* of mailing lists since the early nineties: those which do munge the header are more painful, and routinely have "oops, that was supposed to be a private reply" messages show up. The damage from such messages is *much* higher than the occasional "forget to CC the list" stuff on non-munging lists. Again, this is *opinion,* shared by lots of mailing list maangers / users, but certainly not by all.
at least try to don't threaten us with ban us if we send you private messages by mistake
I don't know what you are talking about here: I didn't threaten to ban anybody. I generally ask (politely) that people keep their replies on-list, and assume that the occasional off-list reply is an honest mistake. I *have* set up filters (in my own mail processing) dropping messages from a couple of users who persist in sending me private mail after being asked to keep the traffic on the list.
There are people that response very rude to a mistake private message
This kind of things are to don't take you seriously, sorry, but this cause a very bad impression of the people of this list
I'm sorry you feel that way.
I respect the opinions, yes, nothing to say about that. I respect you, all of you, sometimes the email is not a good way to express. I'm a little bit critic with some of the decisions you make because I think that Zope is the best operating system (yes, I said operating system) someone could need I think I need to be as critic because when you decide to make a step far the vision of Zope I see more work I need to do for myself I see Zope as a very very smart idea, a univers for my data in a very similar way as a human brain seems to work and when I see how you invest your time in another direction I sufer a lot But, yes, I know that perhaps my vision of Zope is not like yours but it was, at least when I know Zope the people publish some products with the same vision In that point I want to apologize. If I never offend you: sorry, nothing personal
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFJ9cVx+gerLs4ltQ4RAoCdAJ9xOVCuNjNoxf5uMgTeaSQhiaXMNwCgpnvR GHTZGcy1s0hP1/HRJPxX5xE= =shhQ -----END PGP SIGNATURE-----
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
-- Mis Cosas http://blogs.sistes.net/Garito Zope Smart Manager http://blogs.sistes.net/Garito/670
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Garito wrote:
2009/4/27 Tres Seaver <tseaver@palladion.com>
<snip>
at least try to don't threaten us with ban us if we send you private messages by mistake I don't know what you are talking about here: I didn't threaten to ban anybody. I generally ask (politely) that people keep their replies on-list, and assume that the occasional off-list reply is an honest mistake. I *have* set up filters (in my own mail processing) dropping messages from a couple of users who persist in sending me private mail after being asked to keep the traffic on the list.
There are people that response very rude to a mistake private message
Aside from a couple of recent flare-ups, the traffic on this list has been remarkable for its civility over time (I've been reading the list for ten years now). I'm sorry you have experienced rudeness, which I don't find acceptable. I quite like the Ubuntu Code of Conduct[1], and think we could all do to emulate it here.
This kind of things are to don't take you seriously, sorry, but this cause a very bad impression of the people of this list I'm sorry you feel that way.
I respect the opinions, yes, nothing to say about that. I respect you, all of you, sometimes the email is not a good way to express. I'm a little bit critic with some of the decisions you make because I think that Zope is the best operating system (yes, I said operating system) someone could need
I think I need to be as critic because when you decide to make a step far the vision of Zope I see more work I need to do for myself I see Zope as a very very smart idea, a univers for my data in a very similar way as a human brain seems to work and when I see how you invest your time in another direction I sufer a lot
But, yes, I know that perhaps my vision of Zope is not like yours but it was, at least when I know Zope the people publish some products with the same vision
In that point I want to apologize. If I never offend you: sorry, nothing personal
OK, no worries. [1] http://www.ubuntu.com/community/conduct Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ9wVJ+gerLs4ltQ4RAq/qAJ9O2B1nxthBYGoo+NXYGaRrqb98CACfSfsj /w1UTQ97zaKnSlyxx8J+ct8= =mL7R -----END PGP SIGNATURE-----
On Sun, Apr 26, 2009 at 19:45, Tres Seaver <tseaver@palladion.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Patrick Gerken wrote:
Argl, why does this mailing list not use the reply-to: field?
That would be a "religious" question, and one which is not helpful to debate here.. E.g., see:
Thank you for that explanation. That makes sense. Best regards, Patrick
participants (3)
-
Garito -
Patrick Gerken -
Tres Seaver