finding which request dragged all the objects into memory
Hi All, Anyone found any good logging methods or other strategies for determining which request dragged all the object in the zodb into memory? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote:
Anyone found any good logging methods or other strategies for determining which request dragged all the object in the zodb into memory?
I haven't used it this way, but the ActivityMonitor attached to the database keeps track of statistics which could be useful: 'db.getActivityMonitor().log' is the time-series (per request) which is used to generate the data for the "Activity" tab on the Control_Panel, via its 'getActivityAnalysis' method. However, that data doesn't record anything about the request except the "end time". You could also use the connection's 'getTransferCounts' method, which returns the number of loads and stores done on the jar. You might be able to get away with hacking in some extra records into the trace log with this data, e.g.: loads_before, stores_before = connection.getTransferCounts() # do the reqeust loads_after, stores_after = connection.getTransferCounts() from ZServer.DebugLogger import log log('L', id(request), loads_after - loads_before) log('S', id(request), loads_after - stores_before) You might need to monkeypatch Zope2.bobo_before and Zope2.bobo_after to get this hooked in right. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHDkPh+gerLs4ltQ4RAlEvAKC3UHbT0AIyNVGuBV8Inqhh9q3rUgCfYztx wK1DFi+3AdZQ4WQ35jI+beo= =90wc -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote:
Chris Withers wrote:
Anyone found any good logging methods or other strategies for determining which request dragged all the object in the zodb into memory?
I haven't used it this way, but the ActivityMonitor attached to the database keeps track of statistics which could be useful: 'db.getActivityMonitor().log' is the time-series (per request) which is used to generate the data for the "Activity" tab on the Control_Panel, via its 'getActivityAnalysis' method.
However, that data doesn't record anything about the request except the "end time". You could also use the connection's 'getTransferCounts' method, which returns the number of loads and stores done on the jar.
You might be able to get away with hacking in some extra records into the trace log with this data, e.g.:
loads_before, stores_before = connection.getTransferCounts() # do the reqeust loads_after, stores_after = connection.getTransferCounts() from ZServer.DebugLogger import log log('L', id(request), loads_after - loads_before) log('S', id(request), loads_after - stores_before)
You might need to monkeypatch Zope2.bobo_before and Zope2.bobo_after to get this hooked in right.
It was a little tricker than that, but I have a working version available: http://agendaless.com/Members/tseaver/software/thrashcatcher/ Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHDlkt+gerLs4ltQ4RAkJ9AKCog282alWyJMNBUjU122Xt1raoHwCg2UKk o9TZFNzo754Tbsjvw4NAe1M= =RaVw -----END PGP SIGNATURE-----
Hi, I have a story writing application. Users click a creat story button and it creates a file like so... the_folder.manage_addFile(the_id, file="<html></html>", title="", precondition="", content_type="text/html") This works fine the file is created and is of the content type text/html However when the user edits the story using epoz and saves it the file content type changes to application/octet-stream here is the update script called from a dtml method called 'edit' with the url being http://websiteaddress.com/storyid/edit def changeFileContent(ht): ''' this function updates the object contents on which it is called ''' htPlus = "<html><head><title>" + title1 + "</title></head><body>" + ht + "</body></html>" context.manage_upload(htPlus) any ideas? thanks _________________________________________________________________ More photos; more messages; more storageget 5GB with Windows Live Hotmail. http://imagine-windowslive.com/hotmail/?locale=en-us&ocid=TXT_TAGHM_migratio...
Justin Olmanson wrote at 2007-10-13 00:51 +0000:
I have a story writing application. Users click a creat story button and it creates a file like so...
the_folder.manage_addFile(the_id, file="<html></html>", title="", precondition="", content_type="text/html")
This works fine the file is created and is of the content type text/html
However when the user edits the story using epoz and saves it the file content type changes to application/octet-stream here is the update script called from a dtml method called 'edit' with the url being http://websiteaddress.com/storyid/edit
def changeFileContent(ht):
''' this function updates the object contents on which it is called '''
htPlus = "<html><head><title>" + title1 + "</title></head><body>" + ht + "</body></html>"
context.manage_upload(htPlus)
"manage_upload" tries to set the content type. If it gets a "FileUpload" instance as argument, then this instance contains (usually) the content type information and "manage_upload" will use this. Otherwise, it tries to determine the content type from its "__name__" (which means its id) or the content and falls back to "application/octet-stream" if it fails. As you can see, it fails in your case... The code is in "OFS.Image.File._get_content_type". -- Dieter
Chris Withers wrote at 2007-10-11 15:09 +0100:
... Anyone found any good logging methods or other strategies for determining which request dragged all the object in the zodb into memory?
To analyse faults in the persistency design I have instrumented "ZEO.ServerStub" to optionally log its communication with ZEO. Among others, it reports which object it loads, what class and what size the object has. This is not optimal for your current problem but may help a bit.... -- Dieter
participants (4)
-
Chris Withers -
Dieter Maurer -
Justin Olmanson -
Tres Seaver