[Checkins] SVN: keas.pbstate/trunk/s Expanded tests to also serve as docs
Shane Hathaway
shane at hathawaymix.org
Tue Jan 27 18:50:12 EST 2009
Log message for revision 95268:
Expanded tests to also serve as docs
Changed:
U keas.pbstate/trunk/setup.py
U keas.pbstate/trunk/src/keas/pbstate/README.txt
-=-
Modified: keas.pbstate/trunk/setup.py
===================================================================
--- keas.pbstate/trunk/setup.py 2009-01-27 22:57:21 UTC (rev 95267)
+++ keas.pbstate/trunk/setup.py 2009-01-27 23:50:12 UTC (rev 95268)
@@ -11,15 +11,22 @@
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
+"""An object database foundation based on Google's Protocol Buffers"""
+import os
from setuptools import setup
+def read_file(*path):
+ base_dir = os.path.dirname(__file__)
+ file_path = (base_dir, ) + tuple(path)
+ return file(os.path.join(*file_path)).read()
+
setup(
name='keas.pbstate',
version='0.1dev',
author='Shane Hathaway and the Zope Community',
author_email='zope-dev at zope.org',
- description='Object state storage in a Google Protocol Buffer',
+ description=__doc__,
license='ZPL 2.1',
package_dir={'': 'src'},
@@ -29,4 +36,5 @@
'setuptools',
'protobuf',
],
+ long_description = read_file('src', 'keas', 'pbstate', 'README.txt'),
)
Modified: keas.pbstate/trunk/src/keas/pbstate/README.txt
===================================================================
--- keas.pbstate/trunk/src/keas/pbstate/README.txt 2009-01-27 22:57:21 UTC (rev 95267)
+++ keas.pbstate/trunk/src/keas/pbstate/README.txt 2009-01-27 23:50:12 UTC (rev 95268)
@@ -1,20 +1,46 @@
-=====
+Overview
+========
+
+Google's Protocol Buffers project provides an interesting way to serialize
+data. Protocol Buffer messages are efficient to produce and parse, flexible
+enough to weather schema changes, fairly expressive, and are usable in
+several programming languages.
+
+What if we combined those properties with an object database? Object
+databases often provide an excellent software foundation. Unfortunately,
+object databases are generally either joined to a complex object-relational
+mapper, or are bound to a single programming language due to the
+object serialization format. This package uses the latter strategy, but
+uses Protocol Buffers as the serialization format, conceivably making
+it possible to build an object database that multiple programming languages
+can access.
+
+Using this package also provides schema documentation. The Protocol Buffers
+package requires programmers to write the schema of their data in a
+concise form that also serves as documentation of the schema. While it's
+usually possible to guess at the schema by looking at application
+code, having it written out in Protocol Buffer format is much more
+direct and informative.
+
+This package is designed to be combined with an object database such as
+ZODB, but this package does not require ZODB.
+
+
Tests
=====
-These are tests of keas.pbstate, a Python package that provides
-a way to store Python object state in a Google Protocol Buffer.
-These tests also serve as basic documentation of this package.
-This package is designed to be compatible with ZODB, but ZODB is
-not required.
-
+The tests below describe how to use this package.
These tests depend on the module named testclasses_pb2.py, which
-is generated from testclasses.proto using the following command:
+is generated from testclasses.proto using the following command,
+available once the Google Protocol Buffers package is installed::
protoc --python_out . *.proto
-Create a Contact class.
+Create a Contact class. Notice its metaclass. The metaclass adds
+properties to the class so that you can read and write protocol
+buffer message fields using simple attribute access. The
+'create_time' attribute is one such field.
>>> import time
>>> from keas.pbstate.meta import ProtobufState
@@ -27,7 +53,7 @@
...
Create an instance of this class and verify the instance has the expected
-attributes.
+attributes. These attributes are all described in the .proto file.
>>> c = Contact()
>>> c.create_time > 0
@@ -40,7 +66,8 @@
u'United States'
The instance also provides access to the protobuf message, its type (inherited
-from the class), and the references from the message.
+from the class), and the references from the message. References will be
+discussed later.
>>> c.protobuf
<keas.pbstate.testclasses_pb2.ContactPB object at ...>
@@ -81,7 +108,7 @@
------
A class can mix in properties that access sub-messages. This is
-useful when subclassing (although subclassing should be avoided).
+useful when subclassing (although subclassing should be avoided in general).
Here is a class that mixes the ContactPB properties and the AddressPB
properties in a single class.
@@ -102,6 +129,9 @@
Serialization
-------------
+The ProtobufState also provides __getstate__ and __setstate__ methods,
+which Python uses for serialization purposes.
+
Try to serialize the object without providing all of the required fields.
>>> c.__getstate__()
@@ -140,18 +170,20 @@
Classes using the ProtobufState metaclass support references to arbitrary
objects through the use of the 'protobuf_refs' attribute.
-Add a guardian to c2, but don't say who the guardian is yet. Note that
-the .proto file defines the guardians list as a list of messages that
-have a _p_refid attribute.
+Our Contact class has a 'guardians' attribute that contains a list of
+references. The ProtobufState metaclass treats any message or sub-message
+with a _p_refid field as a reference.
+Add a guardian to c2, but don't say who the guardian is yet.
+
>>> guardian_ref = c2.guardians.add()
Call protobuf_refs.set() to make guardian_ref refer to c.
>>> c2.protobuf_refs.set(guardian_ref, c)
-The set method generated a reference ID, then that
-ID was assigned to guardian_ref._p_refid, and the refid and target object
+Let's go over what happened. The set method generated a reference ID, then
+that ID was assigned to guardian_ref._p_refid, and the refid and target object
were added to the internal state of the protobuf_refs instance. Any message
with a _p_refid field is a reference. Every _p_refid field should be
of type uint32.
@@ -183,10 +215,22 @@
Features Designed for ZODB
--------------------------
-The _p_changed attribute, if it exists, is set to True whenever the protobuf
-changes. Here is the PersistentContact class, which has a _p_changed
-attribute.
+This package provides enough features for storing ProtobufState objects
+in ZODB, although without the keas.pbpersist package, the stored objects
+will still be wrapped inside a Python pickle, making them hard for
+languages other than Python to access. See the keas.pbpersist package
+for a straightforward method of storing ProtobufState objects in ZODB
+without using Python pickles.
+In ZODB, objects have a _p_changed attribute to indicate when they
+are dirty. The ProtobufState metaclass causes instances to modify
+the _p_changed attribute if it exists; it is set to True whenever the
+message changes.
+
+Here is a PersistentContact class, which has a _p_changed attribute.
+(We also define a FakePersistent base class in order to avoid
+depending on ZODB.)
+
>>> class FakePersistent(object):
... __slots__ = ('_changed',)
... def _get_changed(self):
@@ -258,8 +302,12 @@
Edge Cases
----------
-Synthesize a refid hash collision. First make a reference:
+Synthesize a refid hash collision. This is a rare occurrence, but
+this package should handle it transparently as long as no single object
+holds more than about one billion (2**30) references to other objects.
+First make a reference:
+
>>> guardian_ref = c2.guardians.add()
>>> c2.protobuf_refs.set(guardian_ref, c)
More information about the Checkins
mailing list