[Zope-dev] Proposal: merge zc.configuration's exclude directive into zope.configuration.

[Martijn Faassen]

Hi there,

On Sat, Feb 28, 2009 at 3:14 PM, Dan Korostelev <nadako at gmail.com> wrote:
> 2009/2/26 Martijn Faassen <faassen at startifact.com>:
[snip]
>> When claims like that are made, I'd like to see measurements that
>> demonstrate significant slowdowns during startup. Undoubtedly more
>> code is excuted than when you write out 'include' directives manually,
>> but is the slowdown actually measurable?
>
> Well, that's probably a really big mis-use (I even hope so), but I ran
> into really gross slowdown while working on the z3ext project. They
> have about 130 eggs (not counting zope 3.4 ones) and each one has the
> autoinclude directive and a bunch of egg dependencies. So the
> application startup time on my (not so fast though) computer is about
> 4-5 mins :-/

Okay, that's a good data point. I haven't used autoinclude on that
scale yet, but I think it *should* be usable on that
scale. How many minutes got added to the startup time when autoinclude
was introduced, any idea?

Some algorithm in the middle of z3c.autoinclude might be slowing
things down, but hopefully a caching approach could help eliminate
this. We need to identify the pain point.

Could you provide some form of profiling data?

> So, is z3c.autoinclude intended to be used by any egg, including
> "feature packages" (like z3c.form) or only for actual applications
> (like a ZMI instance)? I'd really like if this topic will be cleared
> up in z3c.autoinclude docs.

I think it should be useful for any package that has ZCML dependencies
at all, so if that slows things down we should fix it.

> Also, is there any caching for already processed packages in the
> include finder code? If no, I'd probably like to contribute some, if
> I'll use z3c.autoinclude. :)

Ah, you're thinking in the same direction. I don't think there's any
caching at all yet in autoinclude and that'd be the first
thing I'd look at if I were to look into speeding it up. It's just I
hadn't run into the pain point yet.

>>> I'd like to see an option for
>>> packages that are using z3c.autoinclude to make autoincluding
>>> conditional, so those who doesn't like it or needs more control could
>>> just turn it off.
[I ask "why?" not seeing the point]
> Well, it's quite useful in the "z3ext" case I described above - I
> could just turn off the autoincluding and carefully write the includes
> in the instance zcml, so get not so crazy startup time.

Aah, yes, I see that example. The autoinclude trunk, as Ethan said in
another reply, does have support for this with an environment
variable. I didn't see the point for it but now I do. :)

Anyway, I hadn't realized you had more experience with autoinclude
than I thought, I'm sorry for my tone. I hope we can learn from your
feedback to improve the pain points you discovered.

Regards,

Martijn