[Zope3-dev] Zope3+W2k+mingw=Python crash

[Tim Peters]

[Dave Harris]
> I would like to thank everyone for politely ignoring this post and
> not stating the obvious...
>
> Dave Harris is clueless!

That wasn't my impression, but trying to understand an English description
of a detailed code problem on a platform I don't use is proving pretty much
impossible.

> I completely misapprended the problem. Instead of a code problem, the
> abort occurs during maintenance of the garbage collection chain. In
> particular, cm_dealloc() makes the call:
>
>    _PyObject_GC_UNTRACK((PyObject *)cm);
>
> and the process collapses when the assignment to the previous element
> link is attempted. The address of the previous element appears to be
> 0xfffffffd (-3). Now I have to find out why.

Descriptions in terms of the actual C structures and code would really help.
The gc header is declared like so:

typedef union _gc_head {
	struct {
		union _gc_head *gc_next;
		union _gc_head *gc_prev;
		int gc_refs;
	} gc;
	long double dummy;  /* force worst-case alignment */
} PyGC_Head;

By "assignment to the previous element link", I'm betting you mean one of
these two lines from _PyObject_GC_UNTRACK:

	g->gc.gc_prev->gc.gc_next = g->gc.gc_next; \
	g->gc.gc_next->gc.gc_prev = g->gc.gc_prev; \

but I'm not sure which one.  It would be very helpful to know what's in
*all* the members of cm's PyGC_Head header.

Without those details it requires too much pure guessing, but I'm betting
the -3 you're seeing isn't an accident, and am about to milk that for all
it's worth <wink -- but it could be worth a lot!>.

gc.gc_prev is adjacent to gc.gc_refs in the PyGC_Head structure, and the
preprocessor #defines right before the declaration of this struct establish
some special values that appear in gc.gc_refs:

#define _PyGC_REFS_UNTRACKED                    (-2)
#define _PyGC_REFS_REACHABLE	                  (-3)
#define _PyGC_REFS_TENTATIVELY_UNREACHABLE      (-4)

So -3 is _PyGC_REFS_REACHABLE, and is the value we *expect* gc.gc_refs to
have for a tracked object when cyclic gc isn't currently running.  It sounds
like the compiler you're using is confusing it with gc.gc_prev.  This
*could* be because the compiler you're using forces different alignment than
MSVC, perhaps because it has a different belief about what "long double"
means, or maybe just because it uses different alignment rules.  The

	PyGC_Head *g = _Py_AS_GC(o); \

with which _PyObject_GC_TRACK(o) begins is defined as

#define _Py_AS_GC(o) ((PyGC_Head *)(o)-1)

and if your compiler disagrees about sizeof(PyGC_Head), accessing fields of
g won't work as intended.  sizeof(PyGC_Head) is 16 under MSVC.  Under most
32-bit compilers used on Linux, I believes it's 12.  If it's 12 under your
compiler, then I'm betting it looks like so:

                   MSVC         yours
                   -------      -----
    (char*)cm-16   gc_next      -out of bounds-
    (char*)cm-12   gc_prev      gc_next
    (char*)cm-8    gc_refs      gc_prev
    (char*)cm-4    -padding-    gc_refs

and that would explain why a reference to gc_prev compiled by your compiler
actually picks up gc_refs in a PyGC_Head filled in by code compiled under
MSVC.

If that's what's really happening, then I'm afraid you'll have to build
Python with this compiler too (+ extension modules), or find some way to
convince your compiler to use the same alignment rules as MSVC.

There's one other possibility:  Zope's C code is traditionally careless
about following the rules for extension modules, particularly in respecting
the rule that extension modules should never use the macro spellings of
Python C API functions (those are technically reserved for use by the Python
core -- but with some irritating exceptions, which makes the rules darned
hard to remember).

If the above is really what's happening, then if instead of cm_dealloc
doing:

    _PyObject_GC_UNTRACK((PyObject *)cm);

it did what it's supposed to do:

    PyObject_GC_UnTrack((PyObject *)cm);

then none of the *relevant* code would have been compiled by your compiler
(it would have called into the MSVC-compiled PyObject_GC_UnTrack external
function, which (of course) has the same belief about sizeof(PyGC_Head) as
all other MSVC-compiled code).