after reading the Directions Roadmap from DC, I was surprised that a substantial improvement of the searching features of Zope, wasn't mentioned as a major concern.

<snip>

Moreover, everybody uses more Google to look for everything, bypassing windows, doors, and portals!. Why? Because it's terribly smart (not mentioning its 6,000 Linux boxes, by the way), and because there's no need to follow the highly-engineered information architecture of a web site, if there's a trustful shorcut to the relevant content!. So, if I'd have to mention one big feature improvement to Zope, I wouldn't doubt: "search engine". <snip>

On the other hand, Fast, from Norway, already have a nice multimedia search engine, from regular, non-structured, spidered web pages. Can we do that "structuring the unstructured" thing within Zope?

You have posed an important question (and, probably some answers), that hopefully I can clarify. One of the all-important points of the Zope directions document is that our number one goal is to make it wildly easier for _developers_ to create and deploy quality components. Why is this so important? Your questions in this email is why that is so important. You are very interested in high-quality search capabilities, and others certainly are as well. Some other folks care more about E-Commerce, or Corba integration, or communication with Java components. The problem, of course, is that even if DC devoted every single person here to creating the "best search engine" (which we couldn't do for very long - we'd soon be gone), we would still be hard pressed to even come close to making everybody happy or being competitive with every other search engine vendor out there. And the reality is that it is not our goal in life to be a better Google than Google. Multiply that by the number of things people want (ECommerce, Corba, et. al.), and the problem is quite clear - *DC cannot possibly provide the best, most featureful and competitive component for every problem*. The *solution* to this problem is what is outlined in the Zope directions document - dramatically lowering the bar of development to allow a thriving marketplace of robust components (that are *not* written by DC), allows interested parties to write (or better yet, reuse) "the best x component" for their purposes. In the future, Zope may come with "some batteries included", in that a Zope distribution may include the latest versions of the most popular and widely used components. But we hope that the idea of "The ZCatalog" (for instance) will fall by the wayside. Zope may still come with a search component such as ZCatalog that is useful for certain tasks and perhaps as a learningtool, but it will not be an infinitely-scalable infinitely-featureful thing that everyone uses for every problem. The hope is that when you outgrow ZCatalog you can move on to other search components particularly suited to your problem domain. If you scale beyond what ZC can handle, maybe you move up to some VeritySearch component that makes use of existing software. Even now, with the current pain level of component development, building a VeritySearch component would probably take considerably less time than building and maintaining equivalent features into "the ZCatalog". This is the future - the way that Zope will succeed is by being the best framework and component integration platform for the Web, not by trying to compete with verticals like search engine vendors on feature points. "Use the right tool for the job" is something we have always believed in, and providing a platform that will allow you to use and integrate the most appropriate tools will be our focus going forward. That is why "substantial improvement of searching features" is not on the futures roadmap - we do not want to provide the best search engine for every task. We want to make it easy for you to build or integrate the "right" search solution for your task. Brian Lloyd brian@digicool.com Software Engineer 540.371.6909 Digital Creations http://www.digicool.com