[ZODB-Dev] Loading objects directly by OID

[Greg Ward]

On 06 November 2001, Jeremy Hylton said:
> I think we can be pretty sure that you've got a reference to a
> persistent object that is not in the storage.  But I don't think the
> most recent message sheds any light on why.

No, it doesn't.  All I did this morning is determine that stripping the
database in question down to two objects, and throwing away our entire
codebase, does not "fix" the problem.  IOW, it's in the storage now --
it's either a result of the existing codebase or some ZODB oddity.

> What attributes does a WaferDescription have?  Is one a
> DimensionCollection?  In what ways can the methods of a
> WaferDescription modify its state?  Is there any code that modifies a
> WaferDescription, but isn't a method?

Well, in the extreme abstraction I packaged up and mailed out today,
both WaferDescription and DimensionCollection have no instance
attributes, no class attributes, no methods, no superclasses, and no
subclasses:

  class WaferDescription (Persistent):
      pass

  class DimensionCollection (Persistent):
      pass

  sys.modules["mems.run.wafer"].WaferDescription = WaferDescription
  sys.modules["mems.lib.physical_unit"].DimensionCollection = \
    DimensionCollection

Even in this extremely simplified version of our system, the anomalous
behaviour persists: this storage has a reference to an object apparently
not in the storage.

I *think* this simplification rules out the possibility of a
__setstate__() somewhere accidentally creating the mystery object.

In real life, which might still be relevant, things are a tad more
complicated; here are some relevant excerpts from our object schema (as
generated by Grouch; note that class names are fully qualified in
definition lines, but use the obvious shorthand when used to define an
attribute):

  class mems.run.wafer.WaferDescription (mems.lib.base.MXPersistent):
      [...]
      resistivity : PhysicalValue

  class mems.lib.physical_value.PhysicalValue (mems.lib.base.MXBase):
      value : float | RangeValue
      unit : PhysicalUnit

  class mems.lib.physical_unit.PhysicalUnit (mems.lib.base.MXBase):
      dims : DimensionCollection
      name : string
      [...]

  class mems.lib.physical_unit.DimensionCollection (mems.lib.base.MXPersistent):
      dimension_names : [string]
      num_dimensions : int
      dimension_powers : Persistence.PersistentMapping {
                           string : (int, int, int, int, int, int, int) }

Notes:
  * MXPersistent derives from Persistent

  * MXBase does not; PhysicalValue and PhysicalUnit can't be Persistent
    because they use __r{add,mul,...}__(), which don't work with
    ExtensionClass

  * DimensionCollection should be a singleton; this is the class
    that represents the seven fundamental dimensions of the SI (length,
    mass, time, current, temperature, substance amount, and light
    intensity); if there is ever more than one DimensionCollection
    instance in our database, something is wrong.  We recently
    discovered that this was exactly the case, so I wrote a script to
    replace all the accidental duplicates with the One True Instance.
    The fact that one lone DimensionCollection was not caught by this
    script was my tip-off to this referenced-but-not-loadable object.

Let's see, what were your other questions:

> In what ways can the methods of a
> WaferDescription modify its state?

Many ways; instances of this class are definitely mutable.  Scanning the
code, I see mostly direct assignment, list .append(), and method calls
on sub-objects to update them.

> Is there any code that modifies a
> WaferDescription, but isn't a method?

Almost certainly, since it can be updated by our web interface, but
doesn't provide enough set_*() methods.  Generally, the web inteface
does setattr() on objects it's updating directly.  (It's not *supposed*
to do this -- it should use modifier methods -- but people get lazy...)

        Greg