[Zope-Checkins] SVN: Zope/trunk/ Avoid conflict error hotspot in PluginIndexes' Unindex class by using IITreeSets instead of simple ints from the start. Idea taken from ``enfold.fixes``. This optimizes the common case of FieldIndexes being used for a low number of unique values. The edge-case of having a FieldIndex for an unique value per document gets penalized, as it uses many more persistent objects. This really warrants a separate type of index.

[Hanno Schlichting]

Log message for revision 117161:
  Avoid conflict error hotspot in PluginIndexes' Unindex class by using IITreeSets instead of simple ints from the start. Idea taken from ``enfold.fixes``. This optimizes the common case of FieldIndexes being used for a low number of unique values. The edge-case of having a FieldIndex for an unique value per document gets penalized, as it uses many more persistent objects. This really warrants a separate type of index.
  

Changed:
  U   Zope/trunk/doc/CHANGES.rst
  U   Zope/trunk/src/Products/PluginIndexes/common/UnIndex.py

-=-
Modified: Zope/trunk/doc/CHANGES.rst
===================================================================
--- Zope/trunk/doc/CHANGES.rst	2010-10-02 09:57:03 UTC (rev 117160)
+++ Zope/trunk/doc/CHANGES.rst	2010-10-02 09:57:31 UTC (rev 117161)
@@ -26,6 +26,10 @@
 Features Added
 ++++++++++++++
 
+- Avoid conflict error hotspot in PluginIndexes' Unindex class by using
+  IITreeSets instead of simple ints from the start. Idea taken from
+  ``enfold.fixes``.
+
 - Added date range index improvements from ``experimental.catalogqueryplan``.
 
 - Changed policy on handling exceptions during ZCML parsing in ``Products``.

Modified: Zope/trunk/src/Products/PluginIndexes/common/UnIndex.py
===================================================================
--- Zope/trunk/src/Products/PluginIndexes/common/UnIndex.py	2010-10-02 09:57:03 UTC (rev 117160)
+++ Zope/trunk/src/Products/PluginIndexes/common/UnIndex.py	2010-10-02 09:57:31 UTC (rev 117161)
@@ -203,23 +203,20 @@
         """
         indexRow = self._index.get(entry, _marker)
 
-        # Make sure there's actually a row there already.  If not, create
-        # an IntSet and stuff it in first.
+        # Make sure there's actually a row there already. If not, create
+        # a set and stuff it in first.
         if indexRow is _marker:
-            self._index[entry] = documentId
-            # XXX _length needs to be migrated to Length object
+            # We always use a set to avoid getting conflict errors on
+            # multiple threads adding a new row at the same time
+            self._index[entry] = IITreeSet((documentId, ))
+            self._length.change(1)
+        else:
             try:
-                self._length.change(1)
+                indexRow.insert(documentId)
             except AttributeError:
-                if isinstance(self.__len__, Length):
-                    self._length = self.__len__
-                    del self.__len__
-                self._length.change(1)
-        else:
-            try: indexRow.insert(documentId)
-            except AttributeError:
-                # index row is an int
-                indexRow=IITreeSet((indexRow, documentId))
+                # Inline migration: index row with one element was an int at
+                # first (before Zope 2.13).
+                indexRow = IITreeSet((indexRow, documentId))
                 self._index[entry] = indexRow
 
     def index_object(self, documentId, obj, threshold=None):