[Zope-dev] [DB-SIG] SQLite as a Light DB component for Zope and Python

[William Trenker]

--Boundary_(ID_lryJ5XrH0dXYLCRpy7tlcA)
Content-type: text/plain; x-avg-checked=avg-ok-35745422; charset=iso-8859-1;
 format=flowed
Content-transfer-encoding: quoted-printable

At 01:58 AM 4/3/02 +0200, Magnus Lyck=E5 <magnus@thinkware.se> wrote:

>So, we expect to see the announcement of a DB-API 2 complient SQLite=20
>driver any day then! :-)

I wish ! :-)   Actually, I'd love to try it.  But maybe there are some=20
DB-API-2 experienced folks out there who could do this in their sleep (if=20
they ever get any time to sleep).  The SQLite C API is really that simple=20
-- one data structure pointer representing an open database, 3 functions=20
(open_db,close_db,execute_sql)  and 1 callback (to handle the rows in the=20
result set).

Of course I'm being a bit silly -- It takes lots of work to put any robust=
=20
piece of software together.  I'll probably take a stab at it but I wanted=20
to see if the Python/Zope world is even interested.

>I'd say a small subset [of SQL92], or perhaps sideset:

Yet but probably sufficient for a small, local relational data store.

>But it seems a bit closer to SQL than GadFly...and apart from typelessness=
=20
>it seems to support rather extensive SELECT statements. It  might be a=20
>very useful thing I guess.  There are plenty of cases where installing an=
=20
>RDBMS is overkill.

Yes, that's what I thought; for small amounts of data a large, fully=20
featured RDBMS is overkill.  Yet, often, flat tables are not the solution=20
either.   Even for small data stores the benefits of the relational model=20
still apply and flat files can be a real pain when the data relationships=20
are complex.

For instance, as a simple use-case, consider the frequent, common software=
=20
application need for storing configuration data.  Often, this data is=20
stored in a collection of flat files.  The amount of data in these files=20
may be relatively small but the data relationships can still be very=20
complex.  As we all know a collection of flat files doesn't directly=20
provide the mechanism to implement these data relationships.  But most RDB=
=20
software is far too large to justify using this well established data=20
technology for configuration data.  I have often thought it would be=20
beneficial to have a small, low-overhead SQL engine as a software=20
component.  I am suggesting SQLite as a candidate.

>If it lifts entire tables into RAM it might be very memory hungry for=20
>large databases.

This is an excellent and important observation.  One of the reasons I'm=20
suggesting SQLite is that it's memory model is, fortunately, more=20
sophisticated than simply lifting entire tables into RAM.  To quote from=20
The Architecture Of SQLite (http://www.hwaci.com/sw/sqlite/arch.html):

"The B-tree module requests information from the disk in 1024 byte chunks.=
=20
The page cache is responsible for reading, writing, and caching these=20
chunks at the behest of the B-tree module. The page cache also provides the=
=20
rollback and atomic commit abstraction and takes care of reader/writer=20
locking of the database file. The B- tree driver requests particular pages=
=20
from the page cache and notifies the page cache when it wants to modify=20
pages or commit or rollback changes and the page cache handles all the=20
messy details of making sure the requests are handled quickly, safely, and=
=20
efficiently."

>So, how active is the mailing list? And how good? I saw a subscribe link,=
=20
>but no archive (maybe I just missed it.)

When you sign up you are taken to a page that lets you have the option of=20
getting to the archives.  The SQLite mailing list is a Yahoo eGroup forum=20
so to get at the archives you have to sign up with Yahoo -- oh well.

Thanks for commenting,
Bill



"The commandments of the LORD are right, bringing joy to the heart. The=20
commands of the LORD are clear, giving insight to life . . . For this is=20
the love of God, that we keep His commandments. And His commandments are=20
not burdensome." (Psalm 19:8, 1John=20
5:3)    <http://torahteacher.com/>torahteacher.com

--Boundary_(ID_lryJ5XrH0dXYLCRpy7tlcA)
Content-type: text/plain; charset=us-ascii; x-avg=cert;
 x-avg-checked=avg-ok-35745422
Content-transfer-encoding: 7BIT
Content-disposition: inline


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.343 / Virus Database: 190 - Release Date: 3/22/02

--Boundary_(ID_lryJ5XrH0dXYLCRpy7tlcA)--