CMFExternalFile and huge ZODB pickles
Hi, I have a problem with one Zope application and have no clue what's going on. The problem is with one ControlledPage that creates CMFExternalFiles. It seems that each file is somehow included in ZODB pickle so Data.fs grows very fast. Only packing helps. The side effect is that transaction takes quite a long time so conflicts are very common. I tried to recreate the environment on the other machine and everything was fine despite the fact that I used all the same products. The only difference was that original app was running on Linux and I did it on Windows. Any clues? Products used: - ExternalFile 1.2.0 - CMFExternalFile 0.5 - Plone 2.0.5 - Zope 2.7.6 BTW, how do you distinct read and write conflicts in event.log? I only have INFO(0) ZODB conflict error at... Pawel
Pawel Lewicki wrote:
Hi, I have a problem with one Zope application and have no clue what's going on. The problem is with one ControlledPage that creates CMFExternalFiles. It seems that each file is somehow included in ZODB pickle so Data.fs grows very fast. Only packing helps. The side effect is that transaction takes quite a long time so conflicts are very common. I tried to recreate the environment on the other machine and everything was fine despite the fact that I used all the same products. The only difference was that original app was running on Linux and I did it on Windows. Any clues? Products used: - ExternalFile 1.2.0 - CMFExternalFile 0.5 - Plone 2.0.5 - Zope 2.7.6
BTW, how do you distinct read and write conflicts in event.log? I only have INFO(0) ZODB conflict error at...
Read conflicts say "ReadConflict" somewhere. Write conflicts do not. Probably only in recent Zopes though, I doubt 2.7 has that detail. Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Pawel Lewicki wrote:
Any clues? Products used: - ExternalFile 1.2.0 - CMFExternalFile 0.5 - Plone 2.0.5
None of these are products I'd trust. Use a ZODB analysis tool to find out the types of objects you have in your database and see which product is causing the issues......
- Zope 2.7.6
BTW, how do you distinct read and write conflicts in event.log? I only have INFO(0) ZODB conflict error at...
Upgrade to Zope 2.8 cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers napisał(a):
Pawel Lewicki wrote:
Any clues? Products used: - ExternalFile 1.2.0 - CMFExternalFile 0.5 - Plone 2.0.5
None of these are products I'd trust. Use a ZODB analysis tool to find out the types of objects you have in your database and see which product is causing the issues......
Thanx a lot!! I hope you got my mail yesterday. I posted it to group too, but I don't see it here :( Analyze.py gives me that result. I wonder if it's caused by bug in any of the products or script? Class Name Count TBytes Pct AvgSize -------------------------- ------- --------- ----- ------- ... OFS.Folder.Folder 2731 7177565156 95.4% 2628182.04 The only reference to folder I have is: folder.manage_addProduct['ExternalFile'].addExternalFile(filename, '', '',filepath, '',file) Pawel
Pawel Lewicki wrote:
Class Name Count TBytes Pct AvgSize -------------------------- ------- --------- ----- ------- ... OFS.Folder.Folder 2731 7177565156 95.4% 2628182.04
The only reference to folder I have is: folder.manage_addProduct['ExternalFile'].addExternalFile(filename, '', '',filepath, '',file)
Well yes, you're adding an external file to the folder. That changes the folder, so a new copy of it will get written to the ZODB, hence the growth you're seeing. try using a BTreeFolder2 or similar for whatever 'folder' is... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (3)
-
Chris Withers -
Florent Guillaume -
Pawel Lewicki