[Zope-dev] DirectoryStorage 0.03

[Petru Paler]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

---1463811836-866396627-960463732=:7515
Content-Type: TEXT/PLAIN; charset=US-ASCII


   Note that this version (experimentally) uses a dbm database for storing
serials. If you don't have/don't want to use gdbm, replace all occurences
of gdbm with dbhash or dumbdbm. I didn't used anydbm because it has some
strange problems with my python installation :) (but it will be fixed
soon).

Changelog:
0.03:
        - don't die if posix.fsync is not available
        - base class renamed to DirectoryStorage
        - added directory (re)creation and locking
        - added a 'magic' file
        - last oid file open/close bug
        - moved serials to an gdbm database

0.02:
        - converted to using os.path.join

0.01:
        - initial release


-Petru

---1463811836-866396627-960463732=:7515
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="DirectoryStorage.py"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.10.10006081428520.7515@oliver.local.coltronix.com>
Content-Description: 
Content-Disposition: attachment; filename="DirectoryStorage.py"

IyBDb3B5cmlnaHQgKGMpIDIwMDAgUGV0cnUgUGFsZXIgKHBwZXRydUBjb2x0
cm9uaXguY29tKSBBbGwgcmlnaHRzIHJlc2VydmVkLg0KDQoNCiMgVG8tZG8g
bGlzdDoNCiMgQ3JpdGljYWw6DQojICAgIC0gTW9zdCBJL08gb3BlcmF0aW9u
cyBhcmUgbm90IGF0b21pYy4gVGhpcyBtZWFucyB0aGF0IEkvTyBlcnJvcnMg
KGFuZCwgZm9yDQojICAgIHRoYXQgbWF0dGVyLCBhbnkgZXhjZXB0aW9ucyBv
Y2N1cmVkIGluIF9maW5pc2gpIGxlYXZlIHRoZSBzdG9yYWdlIGluIGFuDQoj
ICAgIHVuZGVmaW5lZCBzdGF0ZQ0KIw0KIyBOb3QgY3JpdGljYWw6DQojICAg
IC0gUGVyZm9ybWFuY2UgdHVuaW5nIC0tIGFkZCBjYWNoZXMgd2hlcmUgYXBy
b3BpYXRlDQojICAgIC0gbWFrZSBzdXJlIGxvY2tpbmcgaXMgb2sNCiMgICAg
LSBwcm92aWRlIGRpcmVjdG9yeSBzcGxpdHRpbmcgdG8gZW5hYmxlIHVzZSBv
biBub24tcmVpc2VyIGZpbGVzeXN0ZW1zDQojDQojIC4uLiBhbmQgcHJvYmFi
bHkgbWFueSBvdGhlcnMNCg0KIiIiRGlyZWN0b3J5IHN0b3JhZ2UiIiINCg0K
X192ZXJzaW9uX18gPSAiMC4wMyINCg0KZnJvbSBCYXNlU3RvcmFnZSBpbXBv
cnQgQmFzZVN0b3JhZ2UNCmZyb20gbG9ja19maWxlIGltcG9ydCBsb2NrX2Zp
bGUNCmltcG9ydCBQT1NFeGNlcHRpb24NCmltcG9ydCBvcywgc3RydWN0LCBz
dGF0LCBvcGVyYXRvciwgZ2RibQ0KDQp0cnk6DQoJZnJvbSBwb3NpeCBpbXBv
cnQgZnN5bmMNCmV4Y2VwdDoNCglmc3luYyA9IE5vbmUNCg0KZGVmIG9pZDJz
dHIob2lkKToNCglyZXR1cm4gIiUwMlgiKjggJSBzdHJ1Y3QudW5wYWNrKCIh
OEIiLCBvaWQpDQoNCmRlZiBybXJmKGRpcik6DQoJIlJlY3Vyc2l2ZWx5IGRl
bGV0ZSBkaXIiDQoJZm9yIGYgaW4gb3MubGlzdGRpcihkaXIpOg0KCQlwYXRo
bmFtZSA9IG9zLnBhdGguam9pbihkaXIsIGYpDQoJCW1vZGUgPSBvcy5zdGF0
KHBhdGhuYW1lKVtzdGF0LlNUX01PREVdDQoJCWlmIHN0YXQuU19JU0RJUiht
b2RlKToNCgkJCXJtcmYocGF0aG5hbWUpDQoJCWVsaWYgc3RhdC5TX0lTUkVH
KG1vZGUpOg0KCQkJb3MudW5saW5rKHBhdGhuYW1lKQ0KCQllbHNlOg0KCQkJ
cmFpc2UgJ3JlbW92ZSBlcnJvcicsICdzdHJhbmdlIGZpbGUgaW4gZGlyIHN0
b3JhZ2UnDQoJb3Mucm1kaXIoZGlyKQ0KCQkNCk1BR0lDID0gJ0RpcmVjdG9y
eSBTdG9yYWdlIEZvcm1hdCB2MS4wJw0KDQpjbGFzcyBEaXJlY3RvcnlTdG9y
YWdlKEJhc2VTdG9yYWdlKToNCg0KCWRlZiBfX2luaXRfXyhzZWxmLCBkaXJu
YW1lLCBjcmVhdGU9MCk6DQoNCgkJaWYgbm90IG9zLnBhdGguZXhpc3RzKGRp
cm5hbWUpOg0KCQkJY3JlYXRlID0gMQ0KDQoJCWlmIG9zLnBhdGguZXhpc3Rz
KGRpcm5hbWUpOg0KCQkJdHJ5Og0KCQkJCWYgPSBvcGVuKG9zLnBhdGguam9p
bihkaXJuYW1lLCAnbG9jaycpLCAncisnKQ0KCQkJZXhjZXB0Og0KCQkJCWYg
PSBvcGVuKG9zLnBhdGguam9pbihkaXJuYW1lLCAnbG9jaycpLCAndysnKQ0K
CQkJbG9ja19maWxlKGYpDQoJCQl0cnk6DQoJCQkJZi53cml0ZShzdHIob3Mu
Z2V0cGlkKCkpKQ0KCQkJCWYuZmx1c2goKQ0KCQkJZXhjZXB0Og0KCQkJCXBh
c3MNCgkJCXNlbGYuX2xvY2tfZmlsZSA9IGYNCgkJDQoJCUJhc2VTdG9yYWdl
Ll9faW5pdF9fKHNlbGYsIGRpcm5hbWUpDQoNCgkJc2VsZi5fb2lkX2RpciA9
IG9zLnBhdGguam9pbihkaXJuYW1lLCAnb2lkJykNCgkJc2VsZi5fdGluZGV4
ID0gW10NCgkJc2VsZi5fb2lkID0gJ1wwXDBcMFwwXDBcMFwwXDAnDQoNCgkJ
aWYgY3JlYXRlOg0KCQkJaWYgb3MucGF0aC5leGlzdHMoZGlybmFtZSk6DQoJ
CQkJIyBwcnVuZSB0aGUgb2xkIGRpcg0KCQkJCXJtcmYoZGlybmFtZSkNCg0K
CQkJb3MubWtkaXIoZGlybmFtZSkNCgkJCW9zLm1rZGlyKG9zLnBhdGguam9p
bihkaXJuYW1lLCAnb2lkJykpDQoJCQlmID0gb3Blbihvcy5wYXRoLmpvaW4o
ZGlybmFtZSwgJ2xvY2snKSwgJ3crJykNCgkJCWxvY2tfZmlsZShmKQ0KCQkJ
dHJ5Og0KCQkJCWYud3JpdGUoc3RyKG9zLmdldHBpZCgpKSkNCgkJCQlmLmZs
dXNoKCkNCgkJCWV4Y2VwdDoNCgkJCQlwYXNzDQoJCQlzZWxmLl9sb2NrX2Zp
bGUgPSBmDQoJCQlmID0gb3Blbihvcy5wYXRoLmpvaW4oZGlybmFtZSwgJ21h
Z2ljJyksICd3KycpDQoJCQlmLndyaXRlKE1BR0lDKQ0KCQkJZi5jbG9zZSgp
DQoJCQkNCgkJCXNlbGYuX3NpbmRleCA9IGdkYm0ub3Blbihvcy5wYXRoLmpv
aW4oZGlybmFtZSwgJ3NlcmlhbCcpLCAnbicpDQoJCQlzZWxmLmxvZiA9IG9w
ZW4ob3MucGF0aC5qb2luKGRpcm5hbWUsICdsYXN0X29pZCcpLCAndysnKQ0K
CQllbHNlOg0KCQkJc2VsZi5fc2luZGV4ID0gZ2RibS5vcGVuKG9zLnBhdGgu
am9pbihkaXJuYW1lLCAnc2VyaWFsJyksICd3JykNCgkJCWYgPSBvcGVuKG9z
LnBhdGguam9pbihkaXJuYW1lLCAnbGFzdF9vaWQnKSwgJ3IrJykNCgkJCXNl
bGYuX29pZCA9IGYucmVhZCgpDQoJCQlzZWxmLmxvZiA9IGYNCg0KCWRlZiBf
X2xlbl9fKHNlbGYpOg0KCQlyZXR1cm4gbGVuKG9zLmxpc3RkaXIoc2VsZi5f
b2lkX2RpcikpDQoNCglkZWYgZ2V0U2l6ZShzZWxmKToNCgkJZmlsZXMgPSBv
cy5saXN0ZGlyKHNlbGYuX29pZF9kaXIpDQoJCWZpbGVzID0gbWFwKGxhbWJk
YSB4LGQ9c2VsZi5fb2lkX2Rpcjogb3MucGF0aC5qb2luKGQsIHgpLCBmaWxl
cykNCgkJc2l6ZXMgPSBtYXAobGFtYmRhIHg6IG9zLnN0YXQoeClbc3RhdC5T
VF9TSVpFXSwgZmlsZXMpDQoJCXJldHVybiByZWR1Y2Uob3BlcmF0b3IuYWRk
LCBzaXplcykNCg0KCWRlZiBsb2FkKHNlbGYsIG9pZCwgdmVyc2lvbik6DQoJ
CXNlbGYuX2xvY2tfYWNxdWlyZSgpDQoJCXRyeToNCgkJCWYgPSBvcGVuKG9z
LnBhdGguam9pbihzZWxmLl9vaWRfZGlyLG9pZDJzdHIob2lkKSksICdyYicp
DQoJCQlwID0gZi5yZWFkKCkNCgkJCWYuY2xvc2UoKQ0KCQkJcmV0dXJuIHAs
IHNlbGYuX3NpbmRleFtvaWRdDQoJCWZpbmFsbHk6DQoJCQlzZWxmLl9sb2Nr
X3JlbGVhc2UoKQ0KDQoJZGVmIHN0b3JlKHNlbGYsIG9pZCwgc2VyaWFsLCBk
YXRhLCB2ZXJzaW9uLCB0cmFuc2FjdGlvbik6DQoJCWlmIHRyYW5zYWN0aW9u
IGlzIG5vdCBzZWxmLl90cmFuc2FjdGlvbjoNCgkJCXJhaXNlIFBPU0V4Y2Vw
dGlvbi5TdG9yYWdlVHJhbnNhY3Rpb25FcnJvcihzZWxmLCB0cmFuc2FjdGlv
bikNCg0KCQlpZiB2ZXJzaW9uOg0KCQkJcmFpc2UgUE9TRXhjZXB0aW9uLlVu
c3VwcG9ydGVkLCAiVmVyc2lvbnMgYXJlbid0IHN1cHBvcnRlZCINCg0KCQlz
ZWxmLl9sb2NrX2FjcXVpcmUoKQ0KCQl0cnk6DQoJCQlpZiBzZWxmLl9zaW5k
ZXguaGFzX2tleShvaWQpOg0KCQkJCWlmIHNlcmlhbCAhPSBzZWxmLl9zaW5k
ZXhbb2lkXToNCgkJCQkJcmFpc2UgUE9TRXhjZXB0aW9uLkNvbmZsaWN0RXJy
b3INCgkJCXNlcmlhbCA9IHNlbGYuX3NlcmlhbA0KCQkJc2VsZi5fdGluZGV4
LmFwcGVuZCgob2lkLCBzZXJpYWwsIGRhdGEpKQ0KCQlmaW5hbGx5Og0KCQkJ
c2VsZi5fbG9ja19yZWxlYXNlKCkNCg0KCQlyZXR1cm4gc2VyaWFsDQoNCglk
ZWYgX2JlZ2luKHNlbGYsIHRpZCwgdSwgZCwgZSk6DQoJCXBhc3MNCg0KCWRl
ZiBfY2xlYXJfdGVtcChzZWxmKToNCgkJc2VsZi5fdGluZGV4ID0gW10NCg0K
CWRlZiBfZmluaXNoKHNlbGYsIHRpZCwgdXNlciwgZGVzYywgZXh0KToNCgkJ
dHJ5Og0KCQkJZm9yIG9pZCwgc2VyaWFsLCBkYXRhIGluIHNlbGYuX3RpbmRl
eDoNCgkJCQlzZWxmLl9zaW5kZXhbb2lkXSA9IHNlcmlhbA0KCQkJCXNlbGYu
X3NpbmRleC5zeW5jKCkNCgkJCQlmID0gb3Blbihvcy5wYXRoLmpvaW4oc2Vs
Zi5fb2lkX2RpcixvaWQyc3RyKG9pZCkpLCAnd2InKQ0KCQkJCWYud3JpdGUo
ZGF0YSkNCgkJCQlpZiBmc3luYzoNCgkJCQkJZnN5bmMoZi5maWxlbm8oKSkN
CgkJCQlmLmNsb3NlKCkNCgkJCXNlbGYubG9mLnNlZWsoMCkNCgkJCXNlbGYu
bG9mLndyaXRlKHNlbGYuX29pZCkNCgkJCWlmIGZzeW5jOg0KCQkJCWZzeW5j
KHNlbGYubG9mLmZpbGVubygpKQ0KCQlleGNlcHQ6DQoJCQlyYWlzZSBQT1NF
eGNlcHRpb24uVHJhbnNhY3Rpb25FcnJvciwgJ0kvTyBlcnJvciBjb21taXRp
bmcgdHJhbnNhY3Rpb24gLS0geW91ciBzdG9yYWdlIGlzIGxpa2VseSBjb3Jy
dXB0Jw0KDQoJZGVmIHBhY2soc2VsZiwgdCwgcmVmZXJlbmNlc2YpOg0KCQlt
YXhvaWQgPSBvaWQyc3RyKHNlbGYuX29pZCkNCg0KCQlyb290bCA9IFsnXDBc
MFwwXDBcMFwwXDBcMCddDQoJCXBvcCA9IHJvb3RsLnBvcA0KCQlwaW5kZXgg
PSB7fQ0KCQlyZWZlcmVuY2VkID0gcGluZGV4Lmhhc19rZXkNCgkJd2hpbGUg
cm9vdGw6DQoJCQlvaWQgPSBwb3AoKQ0KCQkJaWYgcmVmZXJlbmNlZChvaWQp
Og0KCQkJCWNvbnRpbnVlDQoNCgkJCSMgc2NhbiBwaWNrbGUgZm9yIHJlZmVy
ZW5jZXMNCgkJCWYgPSBvcGVuKG9zLnBhdGguam9pbihzZWxmLl9vaWRfZGly
LG9pZDJzdHIob2lkKSkpDQoJCQlwID0gZi5yZWFkKCkNCgkJCWYuY2xvc2Uo
KQ0KCQkJcGluZGV4W29pZDJzdHIob2lkKV0gPSBwDQoJCQlyZWZlcmVuY2Vz
ZihwLCByb290bCkNCg0KCQkjIG5vdyBkZWxldGUgYW55IHVucmVmZXJlbmNl
ZCBlbnRyaWVzDQoJCWRlbGV0ZWQgPSBbXQ0KCQlmb3Igb2lkIGluIG9zLmxp
c3RkaXIoc2VsZi5fb2lkX2Rpcik6DQoJCQlpZiBub3QgcmVmZXJlbmNlZChv
aWQpIGFuZCAob2lkIDw9IG1heG9pZCk6DQoJCQkJZGVsZXRlZC5hcHBlbmQo
b2lkKQ0KDQoJCXRyeToNCgkJCWZvciBpIGluIGRlbGV0ZWQ6DQoJCQkJb3Mu
dW5saW5rKG9zLnBhdGguam9pbihzZWxmLl9vaWRfZGlyLGkpKQ0KCQlleGNl
cHQ6DQoJCQlyYWlzZSBQT1NFeGNlcHRpb24uVHJhbnNhY3Rpb25FcnJvciwg
J0kvTyBlcnJvciBkdXJpbmcgcGFjayAtLSB5b3VyIHN0b3JhZ2UgaXMgcHJv
YmFibHkgc2NyZXdlZCB1cCcNCg0K
---1463811836-866396627-960463732=:7515--