[Zope-dev] Massive scalability

[Michael Bernstein]

John Eikenberry wrote:
> 
> 
> Long answer: If you check out the source and/or hit it with the profiler
> you'll see that the way the partial search works is to first do a more
> general search then to limit the hits as much as possible via regex's.
> Both these steps have to happen no matter the batch size, and this is where
> you take the performance hit.

Can you tell us a bit about how many indexes (and what
types) you were maintaining about each object? Another
poster reported no problems with 60,000 objects.

> > So you're saying that you are doing all searching using SQL
> > statements, and not just object retreival and storage,
> > correct? How are you handling full text searches?
> 
> Yes. I'll use MySQL's built in pattern matching facilities. It can do full
> text searches with partial matching, and it can do this fast.  I'm working
> on a system that will return the DataSkin's in responce to the query.
> Allowing me to deal with just the objects yet use all of MySQL's
> facilities.

Sounds interesting.

> I've just started to work on this as part of a larger project, but I'm doing
> it full time and should have something fairly soon. My company is very free
> software friendly, so I'll be able to share it once its ready. If you
> happen to be interested.

Sure, keep me posted.

Thanks,

Michael Bernstein.