[Grok-dev] martian scan "bug" (feature?)

[Chris McDonough]

When martian scans packages (via the ModuleGrokker), eventually it winds up
scanning modules that are defined via .pyc and .pyo file representations that do
not have a corresponding .py source file.  This, at least in my configuration,
leads to errors because these are typically modules left over after, e.g. a
module rename, where the "old" .pyc file sticks around even though its source
file has changed names and has new content.

I *think* what I'd like to do is change martian to (either unconditionally or
optionally) skip modules that only have a .pyc or .pyo representation without a
corresponding .py source.  Below is a strategy to do so, if folks agree.

One strategy to do so would be to use the "inspect.getsourcefile(module)" in
places that try to use "dir(module)" to obtain a list of members of a module.
For example, we might change ModuleGrokker's "grokkers" method to do:

    def grokkers(self, name, module):
        grokker = self._grokker
        # get any global grokkers
        for t in grokker.grokkers(name, module):
            yield t

        # try to grok everything in module
        if inspect.getsourcefile(module) is None:
            raise StopIteration

        for name in dir(module):

The inspect.getsourcefile trick works as so:

With module foo1.py:

import foo2
import inspect
print 'sourcefile ', inspect.getsourcefile(foo2)

With module foo2.py

print 'foo'

If you run foo1.py via "python foo1.py" when foo2.py exists and no foo2.pyc
exists, we get:

foo2
sourcefile  /Users/chrism/projects/martian/src/martian/foo2.py

If you run foo1.py via "python foo1.py" when foo2.py does not exist but foo2.pyc
 does exist, we get:

foo2
sourcefile  None

If you run foo1.py via "python -O foo1.py" when foo2.py exists and no foo2.pyo
exists, we get:

foo2
sourcefile  /Users/chrism/projects/martian/src/martian/foo2.py

If you run foo1.py via "python -O foo1.py" when foo2.py does not exist but
foo2.pyo does exist, we get:

foo2
sourcefile  None

Comments?

- C