[Zope-dev] (ZPatterns) Speeding up Skinscripts

[Steve Spicklemire]

Hi John,

   Makes sense to me! I'm guessing eval was used since it's a little
simpler not to have to keep track of both the string expression and
the compiled expression.. but that's just a guess. However it does
bring up a point I've been wondering about anyway. Now that Ty and
Phillip have moved on to TransWarp, who will be maintaining all the
changes to ZPatterns? SteveA has done a great job of keeping a
modified version available for folks running 2.3.X, but I've seen no
motion to move those changes into the "real" ZPatterns. Now if you
find a great optimization, will it get movedinto ZPatterns too? There
are a number of folks now whove contributed to the 'ZPatterns' project
and have invested significant effort in projects that are based on
ZPatterns, and would like to see it maintained. (me!) I wonder if
there is some way that stewardship for ZPatterns could be either
'handed off' or 'shared' so that these kinds of things can be kept
up-to-date without delaying the promised TransWarp goodies. 

What do you think? 

take care,
-steve

>>>>> "JAE" == John Eikenberry <jae-zdev@kavi.com> writes:

    JAE> We have a fairly large and complex app framework built on
    JAE> ZPatterns. It uses MySQL for storage and the standard
    JAE> Specialist/Rack/DataSkin setup with skinscripts for
    JAE> attributes and triggers.

    JAE> We've found that the speed of getItem is a bit slower than we
    JAE> need. For instance retrieving 200 dataskins takes about 8
    JAE> seconds on a P2-300. After profiling and digging around I
    JAE> think I've found the primary bottleneck. Its the running of
    JAE> eval() on the skinscript's python expression (stored in the
    JAE> Compute class as _fromex and Triggers as callexpr).

    JAE> Note that this becomes the bottleneck after the SQL Method
    JAE> gets cached. The query to the DB takes the most time on the
    JAE> first hit, but after its been cached it takes very little
    JAE> time.

    JAE> The optimization I've been looking at is changing the code
    JAE> from storing a string and eval()ing the string to using
    JAE> compile() at save time and exec() when evaluated.

    JAE> Profiling these 2 ways in little test programs seems to
    JAE> indicate about a 2.5x speedup. Not huge, but combined with
    JAE> better hardware should be enough.

    JAE> But I'm curious why this avenue wasn't taken to begin
    JAE> with. Seems like the way to do it to me. Am I missing
    JAE> something?

    JAE> --

    JAE> John Eikenberry [jae@kavi.com]
    JAE> ______________________________________________________________
    JAE> "A society that will trade a little liberty for a little
    JAE> order will deserve neither and lose both."  --B. Franklin

    JAE> _______________________________________________ Zope-Dev
    JAE> maillist - Zope-Dev@zope.org
    JAE> http://lists.zope.org/mailman/listinfo/zope-dev ** No cross
    JAE> posts or HTML encoding!  ** (Related lists -
    JAE> http://lists.zope.org/mailman/listinfo/zope-announce
    JAE> http://lists.zope.org/mailman/listinfo/zope )