- Zope - Zope lists

RE: [Zope] Analyzing ZODB objects
by Bjorn Stabell 23 Oct '03

23 Oct '03

> From: Dieter Maurer [mailto:dieter@handshake.de] > Bjorn Stabell wrote at 2003-10-22 15:53 +0800: > > What is a good tool for analyzing the size of each object > > in the ZODB, > pickled and unpacked in memory? I'm using Zope 2.6.2. > > I do not think there is such a tool. > > I enhanced the output of "fsdump" to include the pickle size > of object records in a transaction. The main purpose has been > to analyse the reasons for unexpected transaction sizes > (turned out to be ZCatalog Metadata). Would it be possible to deduct anything about the runtime memory footprint of python objects based on their pickle size? I'm just dreaming here; a tool that would be useful in analyzing and optimizing performance of Zope servers, as well as answering questions such as "what the heck is in that ZODB?" :) Something like the database administration tools you have for RDBMS that allow you to browse and change them. For the ZODB, I guess this kind of tool could: - let you browse the database objects, hieararchical tree-like view, perhaps without understanding classes at all, just treating objects as containers of other classes and/or basic python data types. kind-of like a file system view that: - shows the types of objects - shows the contents of objects with basic types - shows the size of objects and total for its subobjects - lets you edit the content types etc - lets you browse transactions (like your fsdump?), single or multiple at a time, and objects changed in transactions in a similar way to above (excluding option to edit objects) This tool could be standalone, or run through Zope. Is this feasible?

2 1

Working with sqlvar type=float
by Mike Doanh Tran 23 Oct '03

23 Oct '03

I used the ZPublisher to convert a variable into a type float(see input field below). I passed the variable "balance" to a ZSQL method where balance was set as type=float. I keep getting an "invalid floating point type error" when i called the updateBal ZSQ method. I even tried to convert balance with _.float(balance) but still get the same error. The mysql db field for balance is set as type float. Can someone help me please? <input type="text" name="rec.balance:float:ignore_empty:records" value=""> --------------------------------- <dtml-in rec mapping> <dtml-call "updateBal(balance=balance,Cust_ID=Cust_ID)"> </dtml-in> -------------------------------- ZSQL Method: updateBal UPDATE customers SET balance=<dtml-sqlvar balance type=float> WHERE Cust_ID=<dtml-sqlvar Cust_ID type=nb> -------------------------------- Thanks, Mike -------------------------------------------------------

3 3

How to call URL with post method from a python script?
by Julian Gollop 23 Oct '03

23 Oct '03

I am writing a python script that needs to call a script on a remote machine (not necessarily another zope script). I have a large amount of data to pass to this remote script, and using the query string format is unfeasible. So how do I do this? Thanks. Julian Gollop Codo Technologies ltd. www.codotech.com www.lasersquadnemesis.com

1 0

Bug in the doc or I missed something
by Gilles Lenfant 23 Oct '03

23 Oct '03

Hi, The zope help in the API/ObjectManagerItem section mentions the "getPhysicalRoot()" that returns the zope root object. In a Python script... x = anyobject.getPhysicalRoot() ...raises an AttributeError, but... x = anyObject.getPhysicalPath() ...works perfectly. When bot methods are supposed to be provided by the same base class !!! Did I miss something or does the help mention a deprecated method (I noticed the problem on 2.6.1 and 2.6.2, I have no older version to check) Thanks in advance. --Gilles

3 3

RE: [Zope] Controlling caching of objects
by Bjorn Stabell 23 Oct '03

23 Oct '03

> From: Toby Dickenson [mailto:tdickenson@geminidataloggers.com] [...] > "Cache Extreme Detail" is the page to study (although it is > too big to post here). Look at what objects are live in each connection. > > > the Top Refcounts, > > That page is good for tracking down memory leaks, but not so good for > analysing cache behaviour. For example, I cant tell whether > those 1628 member objects exist as ghosts (good) or activated objects (bad). I'm a bit surprised; a lot of them were loaded, so that seems to indicate some kind of problem. But it's hard to get something sensible out of the extreme cache detail pages with nearly 10,000 objects loaded, and more ghosted. Is it possible to find out why an object was loaded? Or at least, is it possibly (without searching the entire database) to find out who references an object so you can narrow down the suspects? Regards, -- Bjorn

2 1

VersionLockError ???
by Gilles Lenfant 23 Oct '03

23 Oct '03

Hi, I sometimes got this in random (?) places in a Zope 2.6.1 box Error Type: VersionLockError Error Value: ("'\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\xfe'", '/stuff/versionMenus') The traceback... Traceback (innermost last): Module ZPublisher.Publish, line 102, in publish Module Zope.App.startup, line 200, in commit Module ZODB.Transaction, line 235, in commit Module ZODB.Transaction, line 349, in _commit_objects Module ZODB.Connection, line 391, in commit - __traceback_info__: (('BTrees._IOBTree', 'IOBucket'), '\x00\x00\x00\x00\x00\x00\x04\xfe', '') Module ZODB.FileStorage, line 726, in store VersionLockError: ("'\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\xfe'", '/stuff/versionMenus') Tere's something I can't understand : * The "/stuff" folder has been removed for a long time * This error raises on objects with no connection with that "/stuff", even when browsing ZPT in the Zope help... Is my ZODB corrupted ? How to get rid of this ? Many thanks in advance. --Gilles

2 4

Trainer wanted
by Nicolas Silvestre 23 Oct '03

23 Oct '03

Hello, I'm not sure this list is the right place for this message. I'm looking for a french speaking trainer on different level of Zope. Please contact me as soon as possible. Nicolas -- nicolas.silvestre(a)aliacom.fr www.aliacom.fr

2 1

Webalizer
by Bill Etheredge 23 Oct '03

23 Oct '03

Does anyone have advice for using Webalizer with Zope log files? Can anyone point me to a 'how to' or other guidance? Are there other, perhaps better, options? Thanks, Bill Etheredge The Mesa Group

6 7

RE: [Zope] Controlling caching of objects
by Bjorn Stabell 23 Oct '03

23 Oct '03

> From: Toby Dickenson [mailto:tdickenson@geminidataloggers.com] > > Is that an assumption, or measurement?.... there are several > useful pages in control panel. After restart + 1 minutes (1-2 accesses that does 1-2 catalog queries each) it's at 9,870 objects (total) using 130 MB of RAM, giving 13.1 kB per object. I've got profiling and debugging on now. From experience, it's varied from 8 kB to 13 kB per object. Okay, so what are these objects: CONTROL PANEL / DEBUGGING PAGE Top Refcounts: DateTime.DateTime.DateTime: 59530 BTrees._IIBTree.IISet: 4002 BTrees._IOBTree.IOBucket: 2573 RestrictedPython.compiler_2_1.misc.Set: 2308 AccessControl.User.User: 1632 Products.CMFCore.MemberDataTool.MemberData: 1628 BTrees.OOBTree.OOBucket: 1469 Persistence.PersistenceMapping: 1331 [...] OFS.Image.Image: 692 [...] App.ImageFile.ImageFile: 319 [...] The Cache Detail page corresponds to the Top Refcounts, but doesn't list many classes, such as DateTime.DateTime.DateTime? Hm, that's about all I can find out. Does this look strange? The application running on the site, by the way, is a BBS-type application with a few thousand messages imported from another system. Regards, -- Bjorn

2 1

RE: [Zope] Controlling caching of objects
by Bjorn Stabell 23 Oct '03

23 Oct '03

> From: Toby Dickenson [mailto:tdickenson@geminidataloggers.com] > 15k objects sounds like alot for a catalog query. > > 0.5G/15k = average 33k per object...... that sounds more like > the size of a document object rather than a btree node. > > Both of these facts suggest you might be able to improve > performance by > optimising your application. There are pages in control panel > that will tell you what these 15k objects are. I've been trying to figure that out (hence my other post about browsing the ZODB). Actually, 15,000 target objects per thread. The actual number of objects in memory with 4 threads is around 60,000, so it's more like 8kb per object. Which makes me wonder... [ Sorry for asking so much today, but this is really useful :) ] So you should really try to make sure the entire catalog could fit in memory, or each catalog query will result in activating/deactivating (thrashing) objects? What impacts the amount of memory a catalog query actually uses? The indexes, all of them, loaded in memory? Or only the indexes you use? Or something else? I guess the metadata and the objects themselves won't be touched. Thanks for the clarifying answers, by the way! Regards, -- Bjorn

2 1