[Zope-dev] Re: Zcatalog bloat problem (berkeleydb is a solution?)

[Chris McDonough]

Well,  I'm not sure, unfortunately.  I just wanted to get an idea of
what kinds of indexes you had.  The tranalyzer output doesn't mean too
much to me, because it shows BTree buckets and such getting updated,
which is completely understandable... there are at least two data
structures in the Catalog itself that use a BTree, and each index uses
at least two BTrees.  So it's not all that surprising to see that
output.  What is suprising is to hear the amount of growth a transaction
causes.  The only thing I can think of is that:

a) you're committing inappropriately (at times where it would be OK to
not commit)

b) the data fields your indexing or getting metadata from are large.

c) something awful happened between 2.3.2 and 2.3.3 that I dont
understand.

d) the problem is unrelated to the Catalog.

I'm afraid I can't be any more precise than that.

-C


Giovanni Maruzzelli wrote:
> 
> Hi Chris,
> 
> I don't think this is a problem of ObjectManager, also if it contribute to
> the bloating.
> 
> We do breaks the content in subfolders, but our subfolders easily grows to
> contains some hundred objects.
> 
> Do you think that the number of indexes contribute to the bloating? If this
> is important, we can try to compact them in a littler number (eg: the
> boolean indexes can become a sort of bitmask, eliminate the meta_type,
> etc.).
> 
> This is our indexes (cut and paste from the ZMI), and following there is our
> metadata :
> 
> INDEXES:
>   PrincipiaSearchSource Text Index 2,524
>   autore Keyword Index 4,055
>   bflow0 Field Index 4,055
>   bflow1 Field Index 4,055
>   bflow2 Field Index 4,055
>   bflow3 Field Index 4,055
>   bflow4 Field Index 4,055
>   bflow5 Field Index 4,055
>   bflow6 Field Index 4,055
>   bflow7 Field Index 4,055
>   bflow8 Field Index 4,055
>   bflow9 Field Index 4,055
>   bobobase_modification_time Field Index 4,300
>   dflow0 Field Index 4,055
>   dflow1 Field Index 4,055
>   id Field Index 4,300
>   m_sflow0 Keyword Index 3,960
>   m_sflow1 Keyword Index 3,960
>   m_sflow2 Keyword Index 3,960
>   meta_type Field Index 4,300
>   pseudoId Text Index 4,054
>   revisore Keyword Index 4,055
>   title Text Index 3,844
> 
> METADATA:
> 
>   bobobase_modification_time
>   id
>   meta_type
>   pseudoId
>   title