ZODB error when trying to index object (Input/output error)
Hi I get the following error when certain new objects are added into a plone site - when archetypes tries to set the UID, it asks the catalog for all unique values of the index "UID". I;ve never had input/output errors before from the ZODB. There's definitely enough space on the drive and objects are added fine - it's just that the UID is then not set, which breaks all sorts of Archetypes stuff. Any ideas how I can hunt this one down? Thanks, Felix. 2005-06-20T13:15:26 ERROR(200) ZODB Couldn't load state for 0x111310 Traceback (most recent call last): File "/usr/local/zope/lib/python/ZODB/Connection.py", line 597, in setstate p, serial = self._storage.load(oid, self._version) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 662, in _load h = read(DATA_HDR_LEN) IOError: [Errno 5] Input/output error ------ 2005-06-20T13:15:26 ERROR(200) Archetypes None Traceback (most recent call last): File "/zope/global-products/Archetypes/Referenceable.py", line 69, in _register cid = archetype_tool.registerContent(self) File "/zope/global-products/Archetypes/ArchetypeTool.py", line 681, in registerContent cid = self._genId(object) File "/zope/global-products/Archetypes/ArchetypeTool.py", line 659, in _genId keys = catalog.uniqueValuesFor('UID') File "/usr/local/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 562, in uniqueValuesFor return self._catalog.uniqueValuesFor(name) File "/usr/local/zope/lib/python/Products/ZCatalog/Catalog.py", line 425, in uniqueValuesFor return self.getIndex(name).uniqueValues() File "/usr/local/zope/lib/python/Products/PluginIndexes/common/UnIndex.py", line 445, in uniqueValues return tuple(self._index.keys()) File "/usr/local/zope/lib/python/ZODB/Connection.py", line 597, in setstate p, serial = self._storage.load(oid, self._version) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 662, in _load h = read(DATA_HDR_LEN) IOError: [Errno 5] Input/output error
Felix Ulrich-Oltean wrote:
I get the following error when certain new objects are added into a plone site - when archetypes tries to set the UID, it asks the catalog for all unique values of the index "UID". I;ve never had input/output errors before from the ZODB. There's definitely enough space on the drive and objects are added fine - it's just that the UID is then not set, which breaks all sorts of Archetypes stuff. Any ideas how I can hunt this one down?
This IOError typically indicates a rights problem at the filesystem level. I could also be a full-disk problem or maybe a file corruption. Check the OS-level stuff first, and then try the usual ZODB check/recover stuff. I don't think this is relevant, but it may be useful: http://mail.python.org/pipermail/python-list/2003-March/151633.html --jcc
2005-06-20T13:15:26 ERROR(200) ZODB Couldn't load state for 0x111310 Traceback (most recent call last): File "/usr/local/zope/lib/python/ZODB/Connection.py", line 597, in setstate p, serial = self._storage.load(oid, self._version) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 662, in _load h = read(DATA_HDR_LEN) IOError: [Errno 5] Input/output error ------ 2005-06-20T13:15:26 ERROR(200) Archetypes None Traceback (most recent call last): File "/zope/global-products/Archetypes/Referenceable.py", line 69, in _register cid = archetype_tool.registerContent(self) File "/zope/global-products/Archetypes/ArchetypeTool.py", line 681, in registerContent cid = self._genId(object) File "/zope/global-products/Archetypes/ArchetypeTool.py", line 659, in _genId keys = catalog.uniqueValuesFor('UID') File "/usr/local/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 562, in uniqueValuesFor return self._catalog.uniqueValuesFor(name) File "/usr/local/zope/lib/python/Products/ZCatalog/Catalog.py", line 425, in uniqueValuesFor return self.getIndex(name).uniqueValues() File "/usr/local/zope/lib/python/Products/PluginIndexes/common/UnIndex.py", line 445, in uniqueValues return tuple(self._index.keys()) File "/usr/local/zope/lib/python/ZODB/Connection.py", line 597, in setstate p, serial = self._storage.load(oid, self._version) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 662, in _load h = read(DATA_HDR_LEN) IOError: [Errno 5] Input/output error
-- "Building Websites with Plone" http://plonebook.packtpub.com
Felix Ulrich-Oltean <felix@chaptereight.com> writes:
Hi
I get the following error when certain new objects are added into a plone site - when archetypes tries to set the UID, it asks the catalog for all unique values of the index "UID". I;ve never had input/output errors before from the ZODB. There's definitely enough space on the drive and objects are added fine - it's just that the UID is then not set, which breaks all sorts of Archetypes stuff. Any ideas how I can hunt this one down?
Some further info: I think I've found the culprits - there are 4 records which give errors like this: portal.portal_catalog.uncatalog_object('/sites/cw/data/urbanr') Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 558, in uncatalog_object self._catalog.uncatalogObject(uid) File "/usr/local/zope/lib/python/Products/ZCatalog/Catalog.py", line 411, in uncatalogObject x.unindex_object(rid) File "/usr/local/zope/lib/python/Products/PluginIndexes/PathIndex/PathIndex.py", line 138, in unindex_object self._index[comp][level].remove(docid) File "/usr/local/zope/lib/python/ZODB/Connection.py", line 597, in setstate p, serial = self._storage.load(oid, self._version) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 665, in _load raise CorruptedDataError, h ZODB.FileStorage.CorruptedDataError: I'll try fsrecover on it. Felix.
[Felix Ulrich-Oltean]
... I think I've found the culprits - there are 4 records which give errors like this:
portal.portal_catalog.uncatalog_object('/sites/cw/data/urbanr') Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 558, in uncatalog_object self._catalog.uncatalogObject(uid) File "/usr/local/zope/lib/python/Products/ZCatalog/Catalog.py", line 411, in uncatalogObject x.unindex_object(rid) File "/usr/local/zope/lib/python/Products/PluginIndexes/PathIndex/PathIndex.py", line 138, in unindex_object self._index[comp][level].remove(docid) File "/usr/local/zope/lib/python/ZODB/Connection.py", line 597, in setstate p, serial = self._storage.load(oid, self._version) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 665, in _load raise CorruptedDataError, h ZODB.FileStorage.CorruptedDataError:
and later:
Just to follow up on the original problem, in case someone else has similar troubles. It turned out that the physical disk was broken - I couldn't even copy the files at the os level.
Thank you for the followup! I follow reports of FileStorage corruption keenly, and they never have a thoroughly happy ending -- as it implies on http://zope.org/Wikis/ZODB/FileStorageBackup the only ones that have been resolved in my experience got pinned on HW problems, or catastrophic system software bugs. Often the original reporter doesn't follow up, and then we never know. In some of those cases, I heard later that "the problem went away" after they moved to a different machine. That's consistent with flaky HW, but not conclusive. I appreciate hearing how these turn out -- thank you. "the ZODB guy"
Felix Ulrich-Oltean wrote at 2005-6-20 13:22 +0100:
... 2005-06-20T13:15:26 ERROR(200) ZODB Couldn't load state for 0x111310 Traceback (most recent call last): File "/usr/local/zope/lib/python/ZODB/Connection.py", line 597, in setstate p, serial = self._storage.load(oid, self._version) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 662, in _load h = read(DATA_HDR_LEN) IOError: [Errno 5] Input/output error
This means that your storage file cannot be read. Maybe, it was closed... I think, I saw similar behaviour in our ZEO server when it was shutdown while it was performing an operation in a separated thread ("undoHistory"). The shutdown signal caused the storage to be closed but the separate thread still tried to access the storage. -- Dieter
Felix Ulrich-Oltean <felix@chaptereight.com> writes: Just to follow up on the original problem, in case someone else has similar troubles. It turned out that the physical disk was broken - I couldn't even copy the files at the os level. Felix.
participants (4)
-
Dieter Maurer -
Felix Ulrich-Oltean -
J Cameron Cooper -
Tim Peters