[Zope-dev] R: DBtab and BDBStorage

[Fabio Paracchini]

Hi Andrew,
   here are the answers:

>-----Messaggio originale-----
>Da: Andrew R. Halko [mailto:ahalko@insivia.com]
>Inviato: mercoled=EC 4 giugno 2003 17.02
>A: fparacchini@alteanet.it
>Cc: zope-dev@zope.org
>Oggetto: DBtab and BDBStorage


>Hello Fabio,

>I read your post at:
>http://zope.nipltd.com/public/lists/dev-archive.nsf/AGByKey/35A02E5C64B=
9
>3519 about DBtab and BDBStorage.  I am interested in doing the same
>thing.  I have a Plone site that I expect to become very large and I
>need to find some other solutions than ZODB and its 2GB limit.  Do you
>think you could help me with figuring this out?  I have been reading
>documentation, posts and everything else and am a little confused.  =
Here
>are a few questions I have from reading your post.

>1. What did you need to install to make this work and did you do it
>after you already had a Plone/Zope site going?

There is a document at
http://plone.org/Members/fparacchini/Document.2003-02-14.0631 that =
explains
what you need and what you have to do.

It was last updated mid-February, so now there can be later versions of
pyBSDDB3 and DBTab. Be careful that the right combination of Python
2.1.x/pyBSDDB3/ZODB has to be carefully checked. I got results with  =
the
versions specified in that document, your mileage may vary ;)

>2. What is the code that goes in DBtab.conf, I read through it about =
10
>times and can't understand how I need to set it up?

In DBTab.conf you have to specify a Storage that goes on BDB and a =
Database
that is mounted on a Zope path.

For example, to have the whole site on BDB the very basic configuration =
is:

[Database: Main-BDB]
mount_paths=3D/
open_at_startup=3D1

[Storage: Main-BDB]
type=3DBDBFullStorage

This way you'll create a directory below <zope>/Data/var called =
Main-BDB,
that contains a Berkeley DB with logs.

>3. Do you have to create a new DB manually every time you want to
>separate info, is there no way to setup so that it dynamically adds as
>you need space.

Well, you can also add a storage later. The procedure is a bit involved =
but
not so much:

- I suppose you already have a working DBTab configuration, like the =
one
specified before :)

- Create a folder in Zope/Plone where you will put your data, e.g. =
'/Docs'

- Shutdown Zope/Plone

- Copy the whole data directory (in the example before, it should be in
C:\program files\plone\data\var\Main-BDB, copy it in C:\program
files\plone\data\var\Docs-BDB)

- Edit DBTab.conf, and add a Storage and a Database like this:

[Database: Docs-BDB]
mount_paths=3D/Docs
open_at_startup=3D1

[Storage: Docs-BDB]
type=3DBDBFullStorage

- Start Zope/Plone

- from management interface, delete folder /Docs

- from management interface, create a new 'DBTab mount point'

that should be enough: your /Docs folder should be in the new database,
where it grows independently from the rest of the system. Check =
filesizes on
both directory, to be sure that everything works.

I just have to thank once again Shane Hathaway, for the countless =
answers on
this topic.

I also migrated one DB from FileStorage to BDBStorage, the python =
script I
used, launched directly from python interpreter and not from Zope is =
like
this:

from ZODB.FileStorage import FileStorage
from BDBStorage.BDBFullStorage import BDBFullStorage

DBSrc =3D FileStorage( 'C:\temp\Data.fs', read_only=3D1)
DBDst =3D BDBFullStorage( 'C:\temp\BDB-Data' )

DBDst.copyTransactionsFrom( DBSrc )

DBSrc.close()
DBDst.close()

You can also mix different database flavours, thanks to DBTab. In my
configuration I keep 'main' database on FileStorage, to make it faster =
and
easier to administer, and I created a couple of different DBs with BDB =
for
folders '/Customers' and '/Members'.

>4. I read a little on BerkleyDB Storage, does that give more space =
than
>ZODB?

Yes, you no longer have a 2Gb limit, as in ZODB (at least this is true =
on
every platform, to be honest I don't know if on Linux/Unix you can =
overcome
this limit also with ZODB).

>Thanks for any help on this that you can give.  There should be a
>tutorial on this somewhere; it seems like something a lot of people
>would be interested in.  A 2GB limit is just horrible.  Thanks!

>Andrew R. Halko

I hope that this helps, let me know if you have more suggestions.

Ciao
Fabio