[Grok-dev] martian scan "bug" (feature?)

[Chris McDonough]

Hi Brandon,

Brandon Craig Rhodes wrote:
> Chris McDonough <chrism at plope.com> writes:
> 
>> When martian scans packages (via the ModuleGrokker), eventually it
>> winds up scanning modules that are defined via .pyc and .pyo file
>> representations that do not have a corresponding .py source file.
>> This, at least in my configuration, leads to errors because these are
>> typically modules left over after, e.g. a module rename, where the
>> "old" .pyc file sticks around even though its source file has changed
>> names and has new content.
> 
> The logic you suggest for detecting .pyc files seems similar, but I
> haven't determined if it's identical, to the logic in
> 
>  zope/testing/testrunner.py
> 
> in the "zope.testing" module.  Is it wise for every module that needs to
> detect and eliminate "stale bytecode", as zope.testing so whimsically
> puts it, to have separate logic that needs to be maintained separately?
> Could the logic that you have suggested, Chris, and the logic in the
> zope.testing module, be somehow refactored into a little common toolkit
> so that martian and other Zope packages will all handle stale bytecode
> in the same way?

Well, the zope.testing.testrunner stuff actually *removes* "stale" bytecode.
It's highly unlikely that someone might want to ship an application which uses
Martian but does *not* ship .py files (but does ship .pyc or .pyo files).  This
is a .0001% case, of course, but I'd hate to put logic in there that deleted the
only local copy of a Python module because it did not have a corresponding
source file.

> Because we all know how much Grok needs another dependency. :-) Is there
> maybe a Zope module we're already using where this logic "really
> belongs"?
> 

I've worked up a reasonably simple patch (attached) which fixes the problem.  It
doesn't use the strategy I outline before, because once Martian resolves the
module (in order to use inspect.getsourcefile) it's "too late" to do the right
thing.  Instead, it just does it by ignoring files.

I'll try to write some tests shortly and check it in if no one objects.  It has
the effect that, by default, .pyc and .pyo files are just ignored.  There is no
backwards incompatibility.

- C

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ignore_pyc_or_pyo.patch
Url: http://mail.zope.org/pipermail/grok-dev/attachments/20081119/0dedfd27/attachment.pl