Hi there, On Sat, Feb 28, 2009 at 3:14 PM, Dan Korostelev <nadako@gmail.com> wrote:
2009/2/26 Martijn Faassen <faassen@startifact.com>: [snip]
When claims like that are made, I'd like to see measurements that demonstrate significant slowdowns during startup. Undoubtedly more code is excuted than when you write out 'include' directives manually, but is the slowdown actually measurable?
Well, that's probably a really big mis-use (I even hope so), but I ran into really gross slowdown while working on the z3ext project. They have about 130 eggs (not counting zope 3.4 ones) and each one has the autoinclude directive and a bunch of egg dependencies. So the application startup time on my (not so fast though) computer is about 4-5 mins :-/
Okay, that's a good data point. I haven't used autoinclude on that scale yet, but I think it *should* be usable on that scale. How many minutes got added to the startup time when autoinclude was introduced, any idea? Some algorithm in the middle of z3c.autoinclude might be slowing things down, but hopefully a caching approach could help eliminate this. We need to identify the pain point. Could you provide some form of profiling data?
So, is z3c.autoinclude intended to be used by any egg, including "feature packages" (like z3c.form) or only for actual applications (like a ZMI instance)? I'd really like if this topic will be cleared up in z3c.autoinclude docs.
I think it should be useful for any package that has ZCML dependencies at all, so if that slows things down we should fix it.
Also, is there any caching for already processed packages in the include finder code? If no, I'd probably like to contribute some, if I'll use z3c.autoinclude. :)
Ah, you're thinking in the same direction. I don't think there's any caching at all yet in autoinclude and that'd be the first thing I'd look at if I were to look into speeding it up. It's just I hadn't run into the pain point yet.
I'd like to see an option for packages that are using z3c.autoinclude to make autoincluding conditional, so those who doesn't like it or needs more control could just turn it off. [I ask "why?" not seeing the point] Well, it's quite useful in the "z3ext" case I described above - I could just turn off the autoincluding and carefully write the includes in the instance zcml, so get not so crazy startup time.
Aah, yes, I see that example. The autoinclude trunk, as Ethan said in another reply, does have support for this with an environment variable. I didn't see the point for it but now I do. :) Anyway, I hadn't realized you had more experience with autoinclude than I thought, I'm sorry for my tone. I hope we can learn from your feedback to improve the pain points you discovered. Regards, Martijn