Can't locate PyObject_SetItem
I am trying to track down the cause of a performance problem in ZCTextIndex (OkapiIndex is the leading contender - thanks to Dieter to for the tip!) and have gotten as far as the 'score' routine in okascore.c - which references a routine called 'PyObject_SetItem', for which I cannot locate the source routine. Does anyone know where the source for the 'PyObject_xxx' routines are stored (linux installation)? Thanks in advance, Jonathan
[Jonathan Hobbs]
I am trying to track down the cause of a performance problem in ZCTextIndex (OkapiIndex is the leading contender - thanks to Dieter to for the tip!) and have gotten as far as the 'score' routine in okascore.c - which references a routine called 'PyObject_SetItem', for which I cannot locate the source routine. Does anyone know where the source for the 'PyObject_xxx' routines are stored (linux installation)?
PyXYZ_ABC() functions are part of Python's C API, are supplied by your Python installation, and are documented in the "Python/C API" manual. http://docs.python.org/api/object.html#l2h-208
From: "Tim Peters" <tim.peters@gmail.com>
[Jonathan Hobbs]
I am trying to track down the cause of a performance problem in ZCTextIndex (OkapiIndex is the leading contender - thanks to Dieter to for the tip!) and have gotten as far as the 'score' routine in okascore.c - which references a routine called 'PyObject_SetItem', for which I cannot locate the source routine. Does anyone know where the source for the 'PyObject_xxx' routines are stored (linux installation)?
PyXYZ_ABC() functions are part of Python's C API, are supplied by your Python installation, and are documented in the "Python/C API" manual.
Thanks Tim! I have a better grasp on the situation now, but confusion still reigns... I don't understand how PyObject_SetItem (a python/C API) can increase the size of an IIBucket (which usually uses 'Bucket_grow' from BucketTemplate.c), but I will take it on faith that it magically works somehow, so that I can continue my search for the performance problem with ZCTextIndex! Thanks again, Jonathan
[Jonathan Hobbs] ...
I have a better grasp on the situation now, but confusion still reigns... I don't understand how PyObject_SetItem (a python/C API) can increase the size of an IIBucket (which usually uses 'Bucket_grow' from BucketTemplate.c), but I will take it on faith that it magically works somehow, so that I can continue my search for the performance problem with ZCTextIndex!
Sounds normal. PyObject_SetItem(obj, key, val) is the C way to spell the Python obj.__setitem__(key, val), which is in turn a low-level way to spell the Python "obj[key] = val". Then, in BucketTemplate.c, the BucketType struct's tp_mapping slot is set to the address of Bucket_as_mapping, and Bucket_as_mapping's mp_ass_subscript slot is in turn set to the function bucket_setitem. That means PyObject_Setitem resolves to calling bucket_setitem when called with a bucket obj. bucket_setitem in turn calls _bucket_set, which *may* call Bucket_grow. An excruciating twist at the C level is that setitem functions are used both for setting a key in a mapping, and for deleting a key; the latter occurs if the associated "value" is C's NULL.
From: "Tim Peters" <tim.peters@gmail.com>
[Jonathan Hobbs] ...
I have a better grasp on the situation now, but confusion still reigns... I don't understand how PyObject_SetItem (a python/C API) can increase the size of an IIBucket (which usually uses 'Bucket_grow' from BucketTemplate.c), but I will take it on faith that it magically works somehow, so that I can continue my search for the performance problem with ZCTextIndex!
Sounds normal. PyObject_SetItem(obj, key, val) is the C way to spell the Python obj.__setitem__(key, val), which is in turn a low-level way to spell the Python "obj[key] = val".
Then, in BucketTemplate.c, the BucketType struct's tp_mapping slot is set to the address of Bucket_as_mapping, and Bucket_as_mapping's mp_ass_subscript slot is in turn set to the function bucket_setitem. That means PyObject_Setitem resolves to calling bucket_setitem when called with a bucket obj. bucket_setitem in turn calls _bucket_set, which *may* call Bucket_grow.
An excruciating twist at the C level is that setitem functions are used both for setting a key in a mapping, and for deleting a key; the latter occurs if the associated "value" is C's NULL.
Great explanation! Thanks Tim, that removes most of the magic! Jonathan
participants (2)
-
Jonathan Hobbs -
Tim Peters