[Zope] Q: Zope2 version of "Contributions/DTML Example Folder"?
Joe Grace
occam@serv.net
Sun, 25 Jul 1999 11:07:56 -0700
This is a multi-part message in MIME format.
--------------082AAE28A6FC5FF504EE6F98
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Jens,
doh! Here's the FS which may need other stuff. The install for
Zope2Beta1 was painless, so that's probably (at this point) the easier
way
to upgrade (and get the latest :-). The beta is available at:
http://www.zope.org/Download/Releases/Zope-2.0.0b1
Cheers,
= Joe =
Jens Vagelpohl wrote:
> hi joe,
>
> thanks a lot for providing that bbb.py. the scripts bbb.py and FS.py are
> interrelated, however, and the new bbb.py you sent doesn't like the old
> FS.py. now i'm getting this:
>
> [root@diesel utilities]# python bbb.py -x -f export.zexp export.bbe
> Traceback (innermost last):
> File "bbb.py", line 388, in ?
> if __name__=='__main__': main(sys.argv[1:])
> File "bbb.py", line 381, in main
> rpt=FS.ZEXP(v, file).rpt
> AttributeError: ZEXP
>
> hrmph :(
>
> Jens Vagelpohl
>
> > -----Original Message-----
> > From: Joe Grace [mailto:occam@serv.net]
> > Sent: Sunday, July 25, 1999 13:38
> > To: Jens Vagelpohl; zope@zope.org
> > Subject: Re: [Zope] Q: Zope2 version of "Contributions/DTML Example
> > Folder"?
> >
> >
> > Jens,
> >
> > if you don't see an -x switch, then there must be a difference.
> > I've attached
> > the bbb.py from Zope2b1 for your convenience.
> >
> > = Joe =
> >
> >
> > Jens Vagelpohl wrote:
> >
> > > hm... is there some difference between the bbb.py on alpha3 and
> > beta1? i'm
> > > trying to convert that dtmlExamples export.bbe as well and
> > first off i don't
> > > see an -x switch and secondly i am getting an error message:
> > >
> > > [root@diesel utilities]# python bbb.py -f export.zexp export.bbe
> > >
> > > Corrupted data record at 9
> > > Traceback (innermost last):
> > > File "bbb.py", line 409, in ?
> > > _read_and_report(file, rpt, fromEnd, both, n, show)
> > > File "bbb.py", line 218, in _read_and_report
> > > rpt(None, None, None, None, None, None, None, None, None)
> > > File "/usr/src/Zope-2.0.0a3-src/utilities/FS.py", line 110, in rpt
> > > self.tpc_finish()
> > > File "/usr/src/Zope-2.0.0a3-src/utilities/FS.py", line 173,
> > in tpc_finish
> > > id=self._serial
> > > AttributeError: _serial
> > >
> > > not sure what's happening here :(
> > >
> > > Jens Vagelpohl
> >
--------------082AAE28A6FC5FF504EE6F98
Content-Type: text/plain; charset=us-ascii;
name="FS.py"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="FS.py"
##############################################################################
#
# Zope Public License (ZPL) Version 1.0
# -------------------------------------
#
# Copyright (c) Digital Creations. All rights reserved.
#
# This license has been certified as Open Source(tm).
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
# 1. Redistributions in source code must retain the above copyright
# notice, this list of conditions, and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions, and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
#
# 3. Digital Creations requests that attribution be given to Zope
# in any manner possible. Zope includes a "Powered by Zope"
# button that is installed by default. While it is not a license
# violation to remove this button, it is requested that the
# attribution remain. A significant investment has been put
# into Zope, and this effort will continue if the Zope community
# continues to grow. This is one way to assure that growth.
#
# 4. All advertising materials and documentation mentioning
# features derived from or use of this software must display
# the following acknowledgement:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# In the event that the product being advertised includes an
# intact Zope distribution (with copyright and license included)
# then this clause is waived.
#
# 5. Names associated with Zope or Digital Creations must not be used to
# endorse or promote products derived from this software without
# prior written permission from Digital Creations.
#
# 6. Modified redistributions of any form whatsoever must retain
# the following acknowledgment:
#
# "This product includes software developed by Digital Creations
# for use in the Z Object Publishing Environment
# (http://www.zope.org/)."
#
# Intact (re-)distributions of any official Zope release do not
# require an external acknowledgement.
#
# 7. Modifications are encouraged but must be packaged separately as
# patches to official Zope releases. Distributions that do not
# clearly separate the patches from the original work must be clearly
# labeled as unofficial distributions. Modifications which do not
# carry the name Zope may be packaged in any form, as long as they
# conform to all of the clauses above.
#
#
# Disclaimer
#
# THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS ``AS IS'' AND ANY
# EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DIGITAL CREATIONS OR ITS
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
#
# This software consists of contributions made by Digital Creations and
# many individuals on behalf of Digital Creations. Specific
# attributions are listed in the accompanying credits file.
#
##############################################################################
import struct, tempfile, string, time, pickle, os, sys
from struct import pack, unpack
from cStringIO import StringIO
class FS:
"""FileStorage 'report' writer for converting BoboPOS to FileStorage
"""
def __init__(self, fname, file):
file.seek(0,2)
self._input_size=file.tell()
file.seek(0)
self.__name__=fname
self._file=open(fname,'w+b')
self._file.write('FS21')
self._index={}
self._indexpos=self._index.get
self._tindex=[]
self._tappend=self._tindex.append
self._tfile=tempfile.TemporaryFile()
self._pos=4
self._progress=None
def rpt(self, pos, oid, start, tname, user, t, p, first, newtrans):
if pos is None:
self.tpc_finish()
sys.stderr.write(' %% 100 \n')
return
else:
progress=' %% %.1f \r' % (pos*100.0/self._input_size)
if progress != self._progress:
sys.stderr.write(progress)
self._progress=progress
if newtrans:
try: string.atof(tname)
except:
# Ugh, we have a weird tname. We'll just ignore the transaction
# boundary and merge transactions
if first:
# But we can't ignore the first one, so we'll hack in a
# bogus start date
self.tpc_begin('100', user, t)
else:
if not first: self.tpc_finish()
self.tpc_begin(tname, user, t)
self.store(oid, p)
def store(self, oid, data):
old=self._indexpos(oid, 0)
pnv=None
tfile=self._tfile
write=tfile.write
pos=self._pos
here=tfile.tell()+pos+self._thl
self._tappend(oid, here)
serial=self._serial
data=fixpickle(data, oid)
write(pack(">8s8s8s8sH8s",
p64(oid+1),serial,p64(old),p64(pos),
0,p64(len(data))
)
)
write(data)
def tpc_begin(self, tname, user, desc):
del self._tindex[:] # Just to be sure!
self._tfile.seek(0)
t=string.atof(tname)
y,m,d,h,mi=time.gmtime(t)[:5]
s=t%60
self._serial=struct.pack(
">II",
(((((y-1900)*12)+m-1)*31+d-1)*24+h)*60+mi,
long(s * (1L << 32) / 60)
)
# Ugh, we have to record the transaction header length
# so that we can get version pointers right.
self._thl=23+len(user)+len(desc)
# And we have to save the data used to compute the
# header length. It's unlikely that this stuff would
# change, but if it did, it would be a disaster.
self._ud=user, desc
def tpc_finish(self):
file=self._file
write=file.write
tfile=self._tfile
dlen=tfile.tell()
tfile.seek(0)
id=self._serial
user, desc = self._ud
self._ud=None
tlen=self._thl
pos=self._pos
tl=tlen+dlen
stl=p64(tl)
write(pack(
">8s" "8s" "c" "H" "H" "H"
,id, stl, ' ', len(user), len(desc), 0,
))
if user: write(user)
if desc: write(desc)
cp(tfile, file, dlen)
write(stl)
self._pos=pos+tl+8
tindex=self._tindex
index=self._index
for oid, pos in tindex: index[oid]=pos
del tindex[:]
class ZEXP:
"""Zope Export format 'report' writer
"""
def __init__(self, fname, file):
file.seek(0,2)
self._input_size=file.tell()
file.seek(0)
self.__name__=fname
self._file=open(fname,'w+b')
self._file.write('ZEXP')
self._pos=4
self._progress=None
def rpt(self, pos, oid, start, tname, user, t, p, first, newtrans):
write=self._file.write
if pos is None:
write('\377'*16) # end marker
sys.stderr.write(' %% 100 \n')
return
else:
progress=' %% %.1f \r' % (pos*100.0/self._input_size)
if progress != self._progress:
sys.stderr.write(progress)
self._progress=progress
data=fixpickle(p, oid)
l=len(data)
write(p64(oid+1)+p64(l))
write(data)
self._pos=self._pos+l+16
t32 = 1L << 32
def p64(v, pack=struct.pack):
if v < t32: h=0
else:
h=v/t32
v=v%t32
return pack(">II", h, v)
def cp(f1, f2, l):
read=f1.read
write=f2.write
n=8192
while l > 0:
if n > l: n=l
d=read(n)
write(d)
l = l - len(d)
class Ghost: pass
class Global:
__safe_for_unpickling__=1
def __init__(self, m, n):
self._module, self._name = m, n
def __call__(self, *args):
return Inst(self, args)
def __basicnew__(self):
return Inst(self, None)
def _global(m, n):
if m[:8]=='BoboPOS.':
if m=='BoboPOS.PickleDictionary' and n=='Root':
m='ZODB.conversionhack'
n='hack'
elif m=='BoboPOS.PersistentMapping': m='Persistence'
elif m=='BoboPOS.cPickleJar' and n=='ec':
m=n='ExtensionClass'
else:
raise 'Unexpected BoboPOS class', (m, n)
elif m=='PersistentMapping': m='Persistence'
return Global(m,n)
class Inst:
_state=None
def __init__(self, c, args):
self._cls=c
self._args=args
def __setstate__(self, state): self._state=state
from pickle import INST, GLOBAL, MARK, BUILD, OBJ, REDUCE
InstanceType=type(Ghost())
class Unpickler(pickle.Unpickler):
dispatch={}
dispatch.update(pickle.Unpickler.dispatch)
def load_inst(self):
k = self.marker()
args = tuple(self.stack[k+1:])
del self.stack[k:]
module = self.readline()[:-1]
name = self.readline()[:-1]
klass = _global(module, name)
value=Inst(klass, args)
self.append(value)
dispatch[INST] = load_inst
def load_global(self):
module = self.readline()[:-1]
name = self.readline()[:-1]
klass = _global(module, name)
self.append(klass)
dispatch[GLOBAL] = load_global
def persistent_load(self, oid,
TupleType=type(()), Ghost=Ghost, p64=p64):
"Remap object ids from ZODB 2 stype to ZODB 3 style"
if type(oid) is TupleType:
oid, klass = oid
oid = p64(oid+1), klass
else:
oid = p64(oid+1)
Ghost=Ghost()
Ghost.oid=oid
return Ghost
class Pickler(pickle.Pickler):
dispatch={}
dispatch.update(pickle.Pickler.dispatch)
def persistent_id(self, object, Ghost=Ghost):
if hasattr(object, '__class__') and object.__class__ is Ghost:
return object.oid
def save_inst(self, object):
d = id(object)
cls = object.__class__
memo = self.memo
write = self.write
save = self.save
if cls is Global:
memo_len = len(memo)
write(GLOBAL + object._module + '\n' + object._name + '\n' +
self.put(memo_len))
memo[d] = (memo_len, object)
return
self.save_inst(object._cls)
save(object._args)
memo_len = len(memo)
write(REDUCE + self.put(memo_len))
memo[d] = (memo_len, object)
stuff=object._state
save(stuff)
write(BUILD)
dispatch[InstanceType] = save_inst
def save_global(self, object, name = None):
write = self.write
memo = self.memo
if (name is None):
name = object._name
module=object._module
memo_len = len(memo)
write(GLOBAL + module + '\n' + name + '\n' +
self.put(memo_len))
memo[id(object)] = (memo_len, object)
dispatch[type(Ghost)] = save_global
def fixpickle(p, oid):
pfile=StringIO(p)
unpickler=Unpickler(pfile)
newp=StringIO()
pickler=Pickler(newp,1)
pickler.dump(unpickler.load())
state=unpickler.load()
if oid==-1: state={'_container': state}
pickler.dump(state)
p=newp.getvalue()
return p
--------------082AAE28A6FC5FF504EE6F98--