[ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN
Jim Fulton
jim at zope.com
Tue May 26 11:04:28 EDT 2009
On May 26, 2009, at 10:16 AM, Pedro Ferreira wrote:
>
>>
>> That's what I was afraid of.
>>
>> FileStorage indexes can't be saved after they reach a certain size,
>> where size roughly based on the number of objects.
>>
>> I need to find a way to fix this.
> So, from this I infer that our database has grown in such a
> proportion
> that we're reaching some of the current limits (?) of ZODB.
It's a limit of FileStorage.
> Is there any
> other possible failure related to this that can be foreseen? I mean,
> are
> there any other limits that can cause problems, because of the large
> number of objects? It would be very important for us to know so.
The only other issue is memory size, since FileStorage keeps its index
in memory.
>>>> Also, how many objects are in your database?
>>>>
>>> Hmmm... I have no idea... is there an easy way of calculating that?
>>
>>>>> import ZEO.ClientStorage
>>>>> len(ZEO.ClientStorage.ClientStorage(addr)
>>
>> where addr if the address of your storage server as a host, port
>> tuple.
>>
> So, this returns 19283681. Does this include object revisions?
That's interesting. We have much larger databases.
> In any case, it's not such a surprising number, since we have ~73141
> event objects and ~344484 contribution objects, plus ~492016 resource
> objects, and then each one of these may contain authors, and fore sure
> some associated objects that store different bits of info... So,
> even if
> it doesn't include revisions, 19M is not such a surprising number.
> I've also tried to run the "analyze.py" script, but it returns me a
> stream of '''type' object is unsubscriptable" errors, due to:
>
> classinfo = pickle.loads(record.data)[0]
>
> any suggestion?
No. Unfortunately, most of the "scripts" in ZODB aren't tested or
documented well and tend to bitrot.
> Also, is there any documentation about the basic structures of the
> database available? We found some information spread through different
> sites, but we couldn't find exhaustive documentation for the API
> (information about the different kinds of persistent classes, etc...).
> Is there any documentation on this?
No. Comprehensive ZODB documentation is needed. This is an upcoming
project for me.
Jim
--
Jim Fulton
Zope Corporation
More information about the ZODB-Dev
mailing list