updating homebrew extension of structured text to standard
Hi, i have a ZClass with a property which is renderd as structured text. Because i designed this ZClass before there was a standard for including images, i made my own extension of structured text formating. Short: I used __imageobject__ as markup for the image. Now i want to update all my zclass instances to the new syntax (while upgrading to Zope 2.5.1). I have a Python Script listKBE, which call an External Method updateKBE. If i use it with a path with only a few ZClass instances (about 10) it works (slowly). If i use it with a path with about 550 instances it crashes or is extreme slow. Anyone to suggest a faster method? Thank you, Jochen ---------------------------listKBE-------------------------------------- ## Script (Python) "listKBE" ##bind container=container ##bind context=context ##bind namespace= ##bind script=script ##bind subpath=traverse_subpath ##parameters= ##title= ## results=context.Catalog(meta_type='IPRO Knowledge Base Entry',path='/web/Services/knowledge/KnowledgeBase/TechnischesHandbuch/') for result in results: obj = result.getObject() print obj.id #print obj.propertyItems() #for id,val in obj.propertyItems(): # print id issue = obj.Issue newissue = context.updateKBE(issue) obj.propertysheets.properties.manage_changeProperties({'Issue':newissue}) return printed -------------------updateKBE points to kbeimage_search_replace -------- #!/usr/local/bin/python import string import re def punc_func(exclude): punc = r'' for char in string.punctuation: if char not in exclude: punc = punc + r'\%s' % char return punc digits = string.digits letters = string.letters literal_punc = punc_func("'") dbl_quoted_punc = punc_func("\"") strongem_punc = punc_func('*') under_punc = punc_func('_<>') phrase_delimiters = r'\s\.\,\?\/\!\&\(\)' def kbeimage_search_replace(text): expr = re.compile(r'__([%s%s%s\s]+?)__' % (letters, digits, strongem_punc)).search r=expr(text) if r: #print r start, end = r.span(1) newtext='"bild":img:' newtext=text[:start-2]+newtext+text[start:end]+' '+text[end+2:] #print start,end #print newtext return kbeimage_search_replace(newtext) else: #pass return text -- -------------------------------------------------- Jochen Knuth WebMaster http://www.ipro.de IPRO GmbH Phone ++49-7152-93330 Steinbeisstr. 6 Fax ++49-7152-933340 71229 Leonberg EMail: J.Knuth@ipro.de
Jochen Knuth <jok-zope@ipro.de> wrote:
Hi,
i have a ZClass with a property which is renderd as structured text. Because i designed this ZClass before there was a standard for including images, i made my own extension of structured text formating. Short: I used __imageobject__ as markup for the image.
Now i want to update all my zclass instances to the new syntax (while upgrading to Zope 2.5.1).
I have a Python Script listKBE, which call an External Method updateKBE. If i use it with a path with only a few ZClass instances (about 10) it works (slowly). If i use it with a path with about 550 instances it crashes or is extreme slow. Anyone to suggest a faster method?
Thank you, Jochen
Call get_transaction().commit() every 20 or so objects ? Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com
Hi, Jochen Knuth <jok-zope@ipro.de> wrote:
Hi,
i have a ZClass with a property which is renderd as structured text. Because i designed this ZClass before there was a standard for including images, i made my own extension of structured text formating. Short: I used __imageobject__ as markup for the image.
Now i want to update all my zclass instances to the new syntax (while upgrading to Zope 2.5.1).
I have a Python Script listKBE, which call an External Method updateKBE. If i use it with a path with only a few ZClass instances (about 10) it works (slowly). If i use it with a path with about 550 instances it crashes or is extreme slow. Anyone to suggest a faster method?
Thank you, Jochen
replying to myself: i now do a ZopeFind (or correct: two nested ZopeFinds) in the External Method (attached) and give the result to the converter function. It works now reasonably fast, even with all my ZClass instances (about 600). Is there so much overhead in Python Scripts calling an External method? (i used also ZopeFind in the PythonScript, in a variation of my previous mail) Or is it the getObject() call in the Python Scripts? Ciao, Jochen -- -------------------------------------------------- Jochen Knuth WebMaster http://www.ipro.de IPRO GmbH Phone ++49-7152-93330 Steinbeisstr. 6 Fax ++49-7152-933340 71229 Leonberg EMail: J.Knuth@ipro.de #!/usr/local/bin/python import string import re def punc_func(exclude): punc = r'' for char in string.punctuation: if char not in exclude: punc = punc + r'\%s' % char return punc digits = string.digits letters = string.letters literal_punc = punc_func("'") dbl_quoted_punc = punc_func("\"") strongem_punc = punc_func('*') under_punc = punc_func('_<>') phrase_delimiters = r'\s\.\,\?\/\!\&\(\)' def kbeimage_search_replace(text): expr = re.compile(r'__([%s%s%s\s]+?)__' % (letters, digits, strongem_punc)).search r=expr(text) if r: #print r start, end = r.span(1) newtext='"bild":img:' newtext=text[:start-2]+newtext+text[start:end]+' '+text[end+2:] #print start,end #print newtext return kbeimage_search_replace(newtext) else: #pass return text def updateKBE(self,path): pathobj=self.restrictedTraverse(path) resultbases=self.ZopeFind(pathobj, obj_metatypes=['IPRO Knowledge Base'],search_sub=1) resultlist=[] for fid,fobj in resultbases: results=self.ZopeFind(pathobj, obj_metatypes=['IPRO Knowledge Base Entry']) for id,obj in results: resultlist.append(id) issue = obj.Issue newissue = kbeimage_search_replace(issue) obj.propertysheets.properties.manage_changeProperties({'Issue':newissue}) return resultlist
Hi all,
Jochen Knuth <jok-zope@ipro.de> wrote:
Hi,
i have a ZClass with a property which is renderd as structured text. Because i designed this ZClass before there was a standard for including images, i made my own extension of structured text formating. Short: I used __imageobject__ as markup for the image.
Now i want to update all my zclass instances to the new syntax (while upgrading to Zope 2.5.1).
I have a Python Script listKBE, which call an External Method updateKBE. If i use it with a path with only a few ZClass instances (about 10) it works (slowly). If i use it with a path with about 550 instances it crashes or is extreme slow. Anyone to suggest a faster method?
Thank you, Jochen
again replying to myself:
i now do a ZopeFind (or correct: two nested ZopeFinds) in the External Method (attached) and give the result to the converter function. It works now reasonably fast, even with all my ZClass instances (about 600).
no, it doesn't work :-(
------------------------------------------------------------------------
#!/usr/local/bin/python import string import re
def punc_func(exclude): punc = r'' for char in string.punctuation: if char not in exclude: punc = punc + r'\%s' % char return punc
digits = string.digits letters = string.letters literal_punc = punc_func("'") dbl_quoted_punc = punc_func("\"") strongem_punc = punc_func('*') under_punc = punc_func('_<>') phrase_delimiters = r'\s\.\,\?\/\!\&\(\)'
def kbeimage_search_replace(text):
expr = re.compile(r'__([%s%s%s\s]+?)__' % (letters, digits, strongem_punc)).search r=expr(text) if r: start, end = r.span(1) newtext='"bild":img:' newtext=text[:start-2]+newtext+text[start:end]+' '+text[end+2:] return kbeimage_search_replace(newtext) else: return text
def updateKBE(self,path): pathobj=self.restrictedTraverse(path) resultbases=self.ZopeFind(pathobj, obj_metatypes=['IPRO Knowledge Base'],search_sub=1) resultlist=[] for fid,fobj in resultbases: results=self.ZopeFind(pathobj, obj_metatypes=['IPRO Knowledge Base Entry'])
this should be results=self.ZopeFind(fobj, obj_metatypes=['IPRO Knowledge Base Entry']) not searching recursively else.
for id,obj in results: resultlist.append(id) issue = obj.Issue newissue = kbeimage_search_replace(issue)
and here the Zope process dies on signal 11, if i only print e.g. id, obj.Issue it works, but when i call the kbeimage_search_replace()-function ->kaboom.
obj.propertysheets.properties.manage_changeProperties({'Issue':newissue}) return resultlist
if i test the kbeimage_search_replace()-function manually, i can replace the __image.gif__ several times, but i think that the recursive call is causing the error. Any re expert here, who can change the recursive function into an one pass function (there might be several image refs per issue)? Thank you, Jochen -- -------------------------------------------------- Jochen Knuth WebMaster http://www.ipro.de IPRO GmbH Phone ++49-7152-93330 Steinbeisstr. 6 Fax ++49-7152-933340 71229 Leonberg EMail: J.Knuth@ipro.de
Hi, Jochen Knuth schrieb: [...] if i use re.search in the kbeimage_search_replace function, then Zope will crash with signal 11. if i use the deprecated regex and gsub, then it works. Does anyone know a reason for this behaviour? Ciao, Jochen
#!/usr/local/bin/python import string import re
def punc_func(exclude): punc = r'' for char in string.punctuation: if char not in exclude: punc = punc + r'\%s' % char return punc
digits = string.digits letters = string.letters literal_punc = punc_func("'") dbl_quoted_punc = punc_func("\"") strongem_punc = punc_func('*') under_punc = punc_func('_<>') phrase_delimiters = r'\s\.\,\?\/\!\&\(\)'
not working:
def kbeimage_search_replace(text):
expr = re.compile(r'__([%s%s%s\s]+?)__' % (letters, digits, strongem_punc)).search r=expr(text) if r: start, end = r.span(1) newtext1='"bild":img:' newtext=text[:start-2]+newtext1+text[start:end]+' '+text[end+2:] return kbeimage_search_replace(newtext) else: return text
working:
import ts_regex, regex from ts_regex import gsub ctag_prefix="\([\0- (]\|^\)" ctag_suffix="\([\0- ,.:;!?)]\|$\)" ctag_middl2="[%s][%s]\([^\0- %s][^%s]*[^\0- %s]\|[^%s]\)[%s][%s]"
def kbeimage_search_replace(text): bild = regex.compile(ctag_prefix+(ctag_middl2 % (("_",)*8))+ctag_suffix) s=gsub(bild, '\\1"\\2":img:\\2 \\3',text) return s
def updateKBE(self,path): pathobj=self.restrictedTraverse(path) resultbases=self.ZopeFind(pathobj, obj_metatypes=['IPRO Knowledge Base'],search_sub=1) resultlist=[] for fid,fobj in resultbases: results=self.ZopeFind(fobj, obj_metatypes=['IPRO Knowledge Base Entry']) for id,obj in results: resultlist.append(id) issue = obj.Issue newissue = kbeimage_search_replace(issue) obj.propertysheets.properties.manage_changeProperties({'Issue':newissue}) return resultlist
-------------------------------------------------- Jochen Knuth WebMaster http://www.ipro.de IPRO GmbH Phone ++49-7152-93330 Steinbeisstr. 6 Fax ++49-7152-933340 71229 Leonberg EMail: J.Knuth@ipro.de
Hi Chris, Chris Withers schrieb:
Jochen Knuth wrote:
Does anyone know a reason for this behaviour?
Maybe your re isn't matching what you expect?
i think it is more the thread problem with python on FreeBSD (i forgot to include this information, i use Zope 2.5.1 with Python 2.1.3 on FreeBSD 4.6-Stable). I have not the time to debug this further as this is only a one time conversion and the deprecated regex version works (with warnings). I tested the re version manually and it worked but not within zope.
cheers,
Chris
Ciao, Jochen -- -------------------------------------------------- Jochen Knuth WebMaster http://www.ipro.de IPRO GmbH Phone ++49-7152-93330 Steinbeisstr. 6 Fax ++49-7152-933340 71229 Leonberg EMail: J.Knuth@ipro.de
participants (3)
-
Chris Withers -
Florent Guillaume -
Jochen Knuth