- Zope - Zope lists

RE: [Zope] Help: Zope is extremely slow
by Mayers, Philip J 04 Apr '01

04 Apr '01

That sounds suspiciously like the "Zope dying" problem that a lot of people seem to be having. You aren't by any chance using Postgresql are you? Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Daniel Rusch [mailto:drusch@globalcrossing.com] Sent: 04 April 2001 19:33 To: zope(a)zope.org Subject: [Zope] Help: Zope is extremely slow Our user base has grown significantly in the last few weeks. Unfortunately at hours of peak usage Zope crawls to a halt and won't even serve up pages. we have NUMBER_OF_THREADS=4 would increasing this help? Actually when we I check the processes (with ps -fe | grep py) I only see two the root thread and the thread that runs as nobody? Anybody have any suggests to help the performance of the site? Dan _______________________________________________ Zope maillist - Zope(a)zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )

3 2

RE: [Zope] Help: Zope is extremely slow
by Mayers, Philip J 04 Apr '01

04 Apr '01

I'm rather hoping that Postgres isn't the problem, but I can reliably hang my Zope site using what I think should be fairly reasonable code. Could you try something for me - make a ZSQL query, a PythonScript, and a DTML document, and in the pythonscript do this: print context.standard_html_header(context,context.REQUEST) for v in context.SQL_query(param1='whatever'): print context.dtml_document(context,REQUEST, p1=v.p1, p2=v.p2, p3=v.p3, <etc...> p13=v.p13, p14=v.p14, p15=v.p15, p16=v.p16) print context.standard_html_footer(context,context.REQUEST) return printed The DTML document just has to contain anything: <P>Placeholder text</P> Then call the PythonScript (access it's URL) as *fast* as you can (say as fast a reasonable machine can generate the query). The more fields the SQL query returns (and thus the more named parameters to the DTML document), the easier it is to crash. I get a mix of semi-hangs (child 4 threads dying), hangs (one of the 4 dies, Zope locks up in handling a SIGSTOP as told by "strace -p pid -d", or coredumps with sigsegv). And no, there are no hardware problems. This only happens in very specific circumstances, but I can show you a page where it happens all the time. *Something* is going on, and it's crippling my development. Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Steve Drees [mailto:drees@the-bridge.net] Sent: 04 April 2001 20:15 To: Mayers, Philip J; 'Daniel Rusch'; zope(a)zope.org Subject: RE: [Zope] Help: Zope is extremely slow > That sounds suspiciously like the "Zope dying" problem that a lot > of people > seem to be having. You aren't by any chance using Postgresql are you? Postgres is not necessarily the problem. Zope is solid as a rock at www.wejustrock.com and it is using postgresql.

1 0

Is there a limit to the size of the REQUEST space
by Daniel Rusch 04 Apr '01

04 Apr '01

Is there a limit to the size of the REQUEST space per user and in total. In other words if I have 100 users hit my site can they "use" up the REQUEST space that is available to them??? Dan

1 0

(no subject)
by Wu Man Qi 04 Apr '01

04 Apr '01

Hello, I would like to use CookieCrumbler to authenticate users without using the portal . Where I can find information about it ? Thanks, Man Qi

1 0

Re: [Zope] Building multiple documents from fragments(objects).
by Oliver Bleutgen 04 Apr '01

04 Apr '01

> ... > <div><h3>Build From Loop</h3> > <ul><li>fragment1</li></ul> > </div> > <div><h3>Build From Loop</h3> > <ul><li>fragment2</li></ul> > </div> > ... > Any further hints? substitute <div><h3><dtml-var name="title_or_id"></h3> with <div><h3><dtml-var "_[_['sequence-item']].title_or_id()" ></h3> and don't ask why dtml's syntax is so much nicer than perl ;-> short explanation: _['text'] fetches the object with name text, so _[variable] fetches the object with the name contained in variable. 'sequence-item' is a "forbidden" variable name in python (bc. of the '-'), so you have to write _['sequence-item'] inside the outer brackets above. cheers, oliver

1 0

Re: [Zope] PageTemplates - TAL configuration
by Philip Aylesworth 04 Apr '01

04 Apr '01

>>> Error Type: AttributeError >>> Error Value: _v_blocks >> This is a strange one. I'll have to try installing a setup like >> yours from >> scratch to see if I can replicate it; My CVS checkouts work fine. > I am getting the exact same thing. I have RedHat 7.0 with a fresh > install of Zope 2.3.1 and only TAL 1.0.0 and PageTemplates 1.0.0 > added. I tried previously with ParsedXML-1.0, as well, and got the > same error. Do I need ParsedXML-1.0? > The TAL seems to work fine from the command line using the test > > > script. Well, I tried this again on a different computer with a fresh RedHat 6.2 install, the rest the same as above. Again, I got the same _v_blocks error. Has anyone else seen this? Has anyone gotten TAL working from the standard packages? Any ideas? I really, really want to try TAL! Phil. -- Philip Aylesworth St. Clair College

2 1

Form to file.
by Jason C. Leach 04 Apr '01

04 Apr '01

hi, I have a form and I need the contents, when filled out, to be appended to a coma separated list (line) in a file. What's the best way to do that? Can I do it in the DTML or should it be an External method? j. ...................... ..... Jason C. Leach ... University College of the Cariboo. ..

3 3

Building multiple documents from fragments(objects).
by Alan Young 04 Apr '01

04 Apr '01

Hello All, I have some newbie questions. I'm an experienced perl programmer trying to learn Zope/Python and am trying to do something I know must be possible, but am having a hard time wrapping my brain around this new way of doing things. I have a bunch of document fragments with the following structure: /docs fragment1 fragment2 fragment3 ... doc1/ index_html doc2/ index_html Currently, doc1/index_html and doc2/index_html have the following structure: <dtml-var standard_html_header> <h1><dtml-var title_or_id></h1> <div><h3>fragment1 title</h3> <ul> <li><dtml-var fragment1></li> </ul> </div> <div><h3>fragment2 title</h3> <ul> <li><dtml-var fragment2></li> </ul> </div> <div><h3>fragment3 title</h3> <ul> <li><dtml-var fragment3></li> </ul> </div> ... Is there a way that I can do something like this (warning: It's a clunky mix of perl and dtml): for my $f ( qw( fragment1 fragment2 fragment3 ) ) { <div><h3><dtml-var $f-title</h3> <ul> <li><dtml-var $f></li> </ul> </div> } I would prefer to do this entirely in dtml if at all possible (I need to show non-programmers how to do this if I can make it work and they don't wanna know about Python or Perl stuff). Failing that, is there a way I can do this generically enough that a designer could just use a dtml tag and pass the necessary values? Thanks for your input. Alan

3 5

Malicious HTML remover and HTML to Text converter
by Chris Withers 04 Apr '01

04 Apr '01

Okay, Coupla people asked for this, so here goes: Use as follows: from stripogram import html2text, html2safehtml mylumpofdodgyhtml # a lump of dodgy html ;-) mylumpofcoolcleancollectedhtml = html2safehtml(mylumpofdodgyhtml,valid_tags=('b', 'a', 'i', 'br', 'p')) mylumpoftext = html2text(mylumpofcoolcleancollectedhtml) cheers, Chris PS: Patches to the parser used in html2text greatfully recieved ;-) __doc__ = """HTML filter thanks to Itamar Shtull-Trauring""" import sgmllib, string class HTML2Text(sgmllib.SGMLParser): from htmlentitydefs import entitydefs # replace entitydefs from sgmllib def __init__(self): sgmllib.SGMLParser.__init__(self) self.result = "" self.indent = 0 self.ol_number = 0 def add_line(self,text,newline='\n'): self.result = self.result + self.indent*' ' + text + newline def mod_indent(self,i): self.indent = self.indent + i if self.indent < 0: self.indent = 0 def handle_data(self, data): if data: map(self.add_line,string.split(string.strip(data),'\n')) def unknown_starttag(self, tag, attrs): """ Convert HTML to something meaningful in plain text """ tag = string.lower(tag) if tag[0]=='h' or tag in ['br','pre','p','hr']: # insert a blank line self.add_line('') elif tag =='img': # newline, text, newline src = '' for k, v in attrs: if string.lower(k) == 'src': src = v self.add_line('') self.add_line('Image: %s' % src) elif tag =='li': self.add_line('') if self.ol_number: # num - text self.add_line('%s - ' % self.ol_number,'') self.ol_number = self.ol_number + 1 else: # - text self.add_line('- ','') elif tag in ['dd','dt']: self.add_line('') # increase indent self.mod_indent(+1) elif tag in ['ul','dl','ol']: # blank line #self.add_line('') # increase indent self.mod_indent(+1) if tag=='ol': self.ol_number = 1 def unknown_endtag(self, tag): """ Convert HTML to something meaningful in plain text """ tag = string.lower(tag) if tag[0]=='h' or tag in ['pre']: # newline, text, newline self.add_line('') elif tag =='li': #self.add_line('') pass elif tag in ['dd','dt']: #self.add_line('') # descrease indent self.mod_indent(-1) elif tag in ['ul','dl','ol']: # blank line #self.add_line('') # decrease indent self.mod_indent(-1) self.ol_number = 0 class StrippingParser(sgmllib.SGMLParser): from htmlentitydefs import entitydefs # replace entitydefs from sgmllib def __init__(self): sgmllib.SGMLParser.__init__(self) self.result = "" self.endTagList = [] def handle_data(self, data): if data: self.result = self.result + data def handle_charref(self, name): self.result = "%s&#%s;" % (self.result, name) def handle_entityref(self, name): if self.entitydefs.has_key(name): x = ';' else: # this breaks unstandard entities that end with ';' x = '' self.result = "%s&%s%s" % (self.result, name, x) def unknown_starttag(self, tag, attrs): """ Delete all tags except for legal ones """ if string.lower(tag) in self.valid_tags: self.result = self.result + '<' + tag for k, v in attrs: if string.lower(k[0:2]) != 'on' and string.lower(v[0:10]) != 'javascript': self.result = '%s %s="%s"' % (self.result, k, v) endTag = '</%s>' % tag self.endTagList.insert(0,endTag) self.result = self.result + '>' def unknown_endtag(self, tag): if string.lower(tag) in self.valid_tags: self.result = "%s</%s>" % (self.result, tag) remTag = '</%s>' % tag self.endTagList.remove(remTag) def cleanup(self): """ Append missing closing tags """ for j in range(len(self.endTagList)): self.result = self.result + self.endTagList[j] def html2text(self,s): parser = striphtml.HTML2Text() parser.feed(s) parser.close() return parser.result def html2safehtml(s, valid_tags=('b', 'a', 'i', 'br', 'p')): parser = StrippingParser() parser.valid_tags = valid_tags parser.feed(s) parser.close() parser.cleanup() return parser.result

2 1

How to disable ZODB transaction.
by Juan Carlos Coruña 04 Apr '01

04 Apr '01

Hi zopistas, I'm developing an site that has a few thousands of transactions per day. The site adds and deletes a lot of DTML Documents. The ZODB grows dramatically and when I pack manually the database it consumes a lot of time. Is there a way to disable the transactions persistence in ZODB? When not, how can I invoke a "pack" from a python script to call it periodically with a cron job?

3 2