From: Chris Withers <chrisw@nipltd.com>

I still don't get this :(

Here are the two and a half rules for deriving acquisition from a dotted expression: 1. A.B == (B o A), where A is an unwrapped object 2. (self o parent).B == a. (self.B o (self o parent)), if B is found in 'self' b. (parent.B o (self o parent)), if B is found in 'parent'

A has attributes B and C, C has attribute D

A.B.C.D = (B o A) : .C.D by rule 1

now, since C is found in A:

= ((C o A) o (B o A)) : .D

by rule 2b, then rule 1 = ((C o A).D o ((C o A) o (B o A))) by rule 2a = ((C.D o (C o A)) o ((C o A) o (B o A))) by rule 2a = (((D o C) o (C o A)) o ((C o A) o (B o A))) by rule 1 and then simplification, applied upwards, gives: = ((D o (C o A)) o ((C o A) o (B o A))) = (D o ((C o A) o (B o A)))

...

In this case aq_inner is the entire expression, since aq_self is not a wrapper.

I still don't get this at all :(

If A==(self o parent), then If self is an unwrapped object, then aq_inner(A) == A else aq_inner(A) == aq_inner(self) In other words, aq_inner is the left-inner-most wrapper. Find the leftmost 'o' in the expression, and you've got it. Cheers, Evan @ digicool & 4-am

From: Chris Withers <chrisw@nipltd.com>

Who decided these rules and why are they like they are?

They follow directly from the way the Acquisition machinery works, which was set up by Jim Fulton.

So, is the working below now correct? (I feel like I'm back in School doing maths ;-)

Yes.

Where is the order of simplification defined?

In the Acquisition machinery, which simplifies upwards because that's the way it constructs the wrappers. Cheers, Evan @ digicool & 4-am

Chris Withers wrote:

Steve Alexander said he was going to have a think, but I haven't heard anything since he mentioned it...

I'll have to start running a Kibo-bot soon :-) I've got as far as defining the difference between the two acquisition behaviours. Warning: The following goes into some detail about acquisition algebras. There are lots of nested parentheses with letters in them. Let's say we have the following object tree. The upper-case letters represent names, not identity. The lower-case letters represent identity. / |-A a | |-B b | |-X x | |-P p1 | |-Y y | |-P p0 Let's consider the path X.Y.A.B.P in both types of acquisition. Containment first: Start at the root. Look for X. Find X at x. Look for Y in x. Find Y at y. Look for A in y. Not found, so look for A in x. Not found, so look for A in /. Found as a. Look for B in a. Found as b. Look for P in b. Not found, so look for P in a. Not found, so look for P in /. Found as p0. Context first Start at the root. Look for X. Find X at x. Look for Y in x. Find Y at y. Look for A in y. Not found, so look for A in x. Not found, so look for A in /. Found as a. Look for B in a. Found as b. Look for P in b. Not found, so look for P in a. Not found, so look for P in y. Not found, so look for P in x. Found as p1. The acquisition algebra is based on the "of" operator: (object of context). The difference in algebra is that with containment-first, as you build up the expression, you add to it using the last object you searched as the context part of the expression. With conext-first, you need to add the whole expression so far as the context part of the expression. Containment first: Start at the root. (X of /) Look for X. Find X at x. (Y of X) of (X of /) Look for Y in x. Find Y at y. (A of /) of ((Y of X) of (X of /)) Look for A in y. Not found, so look for A in x. Not found, so look for A in /. Found as a. (B of A) of ((A of /) of ((Y of X) of (X of /))) Look for B in a. Found as b. (P of /) of ((B of A) of ((A of /) of ((Y of X) of (X of /)))) Look for P in b. Not found, so look for P in a. Not found, so look for P in /. Found as p0. Context first Start at the root. (X of /) Look for X. Find X at x. (Y of (X of /)) of (X of /) Look for Y in x. Find Y at y. (A of (Y of (X of /))) of ((Y of (X of /)) of (X of /)) Look for A in y. Not found, so look for A in x. Not found, so look for A in /. Found as a. (B of (A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) of ((A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) Look for B in a. Found as b. (P of (B of (A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) of ((A of (Y of (X of /))) of ((Y of (X of /)) of (X of /)))) of (B of (A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) of ((A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) Look for P in b. Not found, so look for P in a. Not found, so look for P in y. Not found, so look for P in x. Found as p1. I just put that in to give Chris a fright :-) You can see from this that the algorithm is more complex. Acrually, it can be rather simplified, because what I've written out in full above contains many internal repetitions. In C terms, these become pointers to the same acquisition wrapper. So the following... (P of (B of (A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) of ((A of (Y of (X of /))) of ((Y of (X of /)) of (X of /)))) of (B of (A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) of ((A of (Y of (X of /))) of ((Y of (X of /)) of (X of /))) ...is really this: ((P of q) of q) where q is ((B of r) of r) where r is ((A of s) of s) where s is ((Y of t) of t) where t is (X of /) I also wonder whether we need the second part of all the context-first expansions above. That is, whether (P of q) will do, instead of ((P of q) of q). To see if the simplification works, let's take the example of acquiring X.Y in the tree: / |-X |-Y Containment first: (X of /) (Y of X) of (X of /) Simplified Context first: (X of /) (Y of (X of /)) Applying this simplification to my larger example above, we get for the following: Simplified Context first Start at the root. (X of /) Look for X. Find X at x. (Y of (X of /)) Look for Y in x. Find Y at y. (A of (Y of (X of /))) Look for A in y. Not found, so look for A in x. Not found, so look for A in /. Found as a. (B of (A of (Y of (X of /))) Look for B in a. Found as b. (P of (B of (A of (Y of (X of /))) Look for P in b. Not found, so look for P in a. Not found, so look for P in y. Not found, so look for P in x. Found as p1. The search just proceeds as you read from left to right. There are no repeated searches, unless you specify a repeated search (such as A.B.A.A.B.C). Therefore, there is little need to simplify expressions in the simplified context-first algebra. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net

Michel Pelletier has reminded me that we need a security check by containment at each point in the search through context. So, in my example, the search will look like this: / |-A a | |-B b | |-X x | |-P p1 | |-Y y | |-P p0 Context-first Acquisition (P of (B of (A of (Y of (X of /))) Security p0 / b a / a b y x / x / / Checks That makes for a lot of security checks. There are possible optimisations, though. But this starts to get even more complicated. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net

Chris Withers wrote:

Steve Alexander wrote:

...

That makes for a lot of security checks. There are possible optimisations, though. But this starts to get even more complicated.

Does that mean it won't work, would be very slow, or both? ;-)

It will work. It will be slower. I think Evan Simpson's suggestion of adding two new methods to the acquisition wrapper is a good idea. That way, Zope remains backward-compatible, and you get to make an explicit choice of how you want acquisition to work, if you wish. Perhaps there's a Collector entry on this? ---- quoted from Evan's email. Untested. def aq_context(ob): context = [] while ob is not None: context.append(ob.aq_base) ob = ob.aq_parent ob = context.pop() while context: ob = context.pop().__of__(ob) return ob def aq_containment(ob): context = [] while ob is not None: context.append(ob.aq_base) ob = ob.aq_inner.aq_parent ob = context.pop() while context: ob = context.pop().__of__(ob) return ob ---- As he pointed out, they'd make useful external methods too. Perhaps these will find their way into Zope 2.3? -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net

Michel Pelletier wrote:

...

A.B.C.D

Look for D in C, if it's not there, look in B if it's not there, look in A

Someone correct me if I'm wrong here, but...

This is a linear search, which is the way acquisition *used* to work.

Which Zope version? ;-)

Consider:

A / B/ C/

D

(A contains B, B contains C and D).

A.B.C.D

By your desire first look for D in C, then B and then A.

Yup...

But A says "You cannot see Ds". Now, in the linear case, D is found in B, and A is never asked if this operation is permitted. This is a security violation because root policies should be enforced unless they are explicitly overidden further down.

Hurm :S I didn't say I had the answers, I was just posing the question ;-)

Further, it actually makes *more* sense because the

Well, maybe for security, but not for everyday use by people who have trouble getting their heads around (((D o C) o (C o A)) o ((C o A) o (B o A))) and the like...

Not from the perspective of the way most development is done. For example, when you want to have your children objects (whatever they may be) be acquireable, you don't need to think about the complexity of the situation, you just make sure you make them accessable in the context *of* you.

Hmmm, the thing which is bugging me is described quite nicely here: http://www.zope.org/Wikis/zope-dev/AcquisitionFeedback How can Steve achieve what he wants with the way acquisition currently works? cheers, Chris

Evan Simpson wrote:

I haven't got the whole reason, but here are some of the pieces:

- never expose a "bare" object, or even one with an incomplete context

Why? You can get at it through aq_base anyway, surely? Also, Jim Fulton wrote a while back:

Sure. There is no guarentee that all objects are wrapped.

- allow the user to backtrack along the context chain

I take it this is the reason for wrappers? Would it matter if the wrappers were structured differently to provide a different search order?

These two together give you the part about aq_parent always being the previous acquisition result.

Yup, got that :-)

- allow recovery of containment information

Urm, didn't quite follow that :S

- base security on containment

aq_inner, right?

These two motivate the simplification of raw acquisition.

Which simplification is that?

The current acquisition implementation is thus a weird hybrid of containment and context.

I'll say ;-)

It isn't hard to convert a standard acquisition wrapper into either of the other sort. I'm going to propose adding something like the following functions: def aq_context(ob): def aq_containment(ob):

Okay, so taking our long-running example:

A / I B/ I C/ D

(A contains B and C, C contains D). (A contains an I, as does B) A.B.C.D

Which I, if any, would each of the following return: <dtml-var expr="aq_context(A.B.C.D).I"> <dtml-var expr="aq_context(A.B.C).I"> <dtml-var expr="aq_context(A.B).I"> <dtml-var expr="aq_context(A).I"> <dtml-var expr="aq_containment(A.B.C.D).I"> <dtml-var expr="aq_containment(A.B.C).I"> <dtml-var expr="aq_containment(A.B).I"> <dtml-var expr="aq_containment(A).I"> cheers, Chris

Evan Simpson wrote:

...

...

- never expose a "bare" object, or even one with an incomplete context

Why? You can get at it through aq_base anyway, surely?

Only from unrestricted code. DTML and (CVS) Python Methods only let you access aq_parent. This only applies to objects that are part of the containment hierachy, of course. Brand new objects aren't wrapped, nor are simple non-persistent types like lists and dicts.

Ah, okay... so really, prettymuch all Zope objects should be wrapped with only one or two exceptions, most of which aren't actually objects as such but simple data types?

...

Would it matter if the wrappers were structured differently to provide a different search order?

Only to anyone who depends on the current behavior :-)

Heh, ain't backward compatability a bitch ;-)

Also, there is a very limited range of "natural" ways to construct the wrappers. Once contructed, of course, we can fool with them in arbitrary ways.

So I see, does this reduce efficiency at all?

We want to be able to find out what an object's container is, regardless of what we acquired it from. In raw acquisition, there's no straightforward way to do this. In simplified acquisition, aq_inner.aq_parent is the container. The simplification is the rule (A o B) o (B o C) => A o (B o C).

I don't see how this simplification actually makes a difference... In what way does it change the search order?

From what I read, I though it just reduced the number of checks you had to do...

All of these would search the dotted expression from right to left, so they would give you B's I.

...

<dtml-var expr="aq_context(A.B.C.D).I"> <dtml-var expr="aq_context(A.B.C).I"> <dtml-var expr="aq_context(A.B).I">

cool :-)

All of these would search the containers of the right-most object, so the first two would give you A's I, and the third B's I.

...

<dtml-var expr="aq_containment(A.B.C.D).I"> <dtml-var expr="aq_containment(A.B.C).I"> <dtml-var expr="aq_containment(A.B).I">

weird ;-S (but I suppose someone might find it useful ;-) The external methods were very cool. How should I got about petitioning for <dtml-var anobject aq_context> to become valid syntax? cheers, Chris

Shane Hathaway wrote:

...

How should I got about petitioning for <dtml-var anobject aq_context> to become valid syntax?

There's one little (okay, big) problem with this idea: aq_context strips the security context. In fact, it could be used to confuse the security machinery.

Let's say I'm Joe Hacker and I have set up membership at www.zope.org/Members/jhacker. I create a DTML method called index_html with this:

<dtml-with Members> <dtml-with hathawsh aq_context> <dtml-call expr="index_html.manage_edit('1 0WN U')"> </dtml-with> </dtml-with>

Alright, I give up :-( This would be really useful, but if it's going to open up security holes everywhere, then I best leave it alone :-S cheers, Chris

Steve Alexander wrote:

Without further optimisation, this means a containment security check for each element of the context. Which kind of suggests worse than linear performance as the context path grows.

Well, I wonder how hard it'd be to implement? (fancy giving it a go? ;-) It may be slow, but as long as it's secure there'd be some really good uses for it where the performance hit might be worthwhile. cheers, Chris