[The main discussion is focused on CMF, please keep zope-cmf@zope.org in the Cc.] Basically we need a search API. I'm Cc-ing zope-dev, exuserfolder-devel and plone-developers (for GRUF) because we're talking about user folders. The last part is especially of interest to you guys. Jens wrote:
Florent Wrote:
Currently CMF's MemberDataTool has searchMemberData(search_param, search_term, attributes=()) but it can only do a search on one attribute. It returns a list of dictionaries.
CMFLDAP's MemberDataTool inherits the one from CMF, and so actually does the search only in cached members. This is a big problem for me.
The way it is done in CMFLDAP stems from how the LDAPUserFolder reacts to calls such as getUsers, which will retrieve all users. Calls like that are inappropriate and even dangerous for any user folder that has a storage with large numbers of records. Zope (and possibly the storage backend used by the user folder) will come to a standstill if all of a sudden thousands of records come flooding back and they all need real Zope user objects created for them.
The LDAPUserFolder has a notion of caching users to avoid the speed penalty caused by having to go to LDAP. I made the conscious decision to implement getUsers in a way that will only look at the currently-cached users. Anything else is potentially disruptive for the running Zope or the LDAP backend.
Yes I understand the reasons for this behavior. Still it's painful, and is a proof we need a search API.
Plone (in MembershipTool) has searchForMembers(**kw) that could search on arbitrary attributes but actually only does it on name, email, roles, and last_login_time. It returns a list of user objects.
CMFLDAP's PloneLDAPMembershipTool seems to be more intelligent than its MemberDataTool and do the full LDAP search, but I'm not sure.
The PloneLDAPMembershipTool's searchForMembers is different in response to specific Plone needs. In my opinion it still sucks really bad. If you look at the code in searchForMembers in the standard Plone membership tool you will see that it does the "dangerous" thing of just grabbing all users from the user folder and then looking at each user to see if the search criteria fit.
There's another item that makes searching members in a CMF-based portal more complicated: Users that you find in the user folder are not necessarily members, even if the user seems to have the Member role. You always end up having to introspect the MemberData tool to see if it has a member data wrapper with the same ID.
Yeah maybe we also need a MembershipTool.hasMember() method ? It's better than doing 'id in mtool.listMemberIds()' ...
I'd like to propose a standardized API on which we can agree for the future: MemberDataTool: searchForMembers(mapping, maybe_some_options=None, **kw) It would search on all the keys of mapping+kw. The options would be there to ask for a substring or exact match. To be decided later. I'd like it to return a list of users, but this may be problematic from a security point of view. What do you think? Otherwise we could make it take an attributes=('email', 'fullname') args, and return a dict with at least the specified attributes.
If you want to go the "sequence of dictionaries" route then the properties as defined on the MemberData Tool should be looked up to define a set of standard attributes to return, meaning attributes above and beyond the user name.
Yes.
A side problem is that this method will have to have knowledge of how to do a search in the user folder. And there are lots of those, with nowhere an standardized API in sight to do search either :-( (actually I don't know any that can do a search appart from LDAPUserFolder). Maybe we could take the oportunity to define one ? Searching for userids, roles, groups (for me), is a prerequisite for me.
A standard API for such searches on the user folders would be a prerequisite for any of this so that the user folder, which is the only link in this chain that really knows how to make the search efficient, can do so. That would completely eliminate that nasty pattern of "I will grab all users myself and then look at each user object".
I agree, so I'd like to propose an API for this too. I haven't thought about it much. LDAPUserFolder has findUser(search_param, search_term, attrs=[]) but it only searches one one term. So I propose: searchUsers(mapping, attributes=(), other_options=..., **kw) it would work in the same manner as searchForMembers above. As above, do we make it return a sequence of user objects or a sequence of dictionaries ? In any case we'll need a way to only return user ids. Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com