September 2004 - Zope-Dev

Possible Windows Service improvements.
by Mark Hammond 12 Oct '04

12 Oct '04

Hi all, I am starting to venture into the wonderful world of Zope! With the benefit of a complete lack of Zope experience, I have been able to look at the Windows service support from a fairly clean slate. However, I also realize this lack of experience means my ideas may be naive - hence I have attempted to split them into discrete issues for discrete rejection <wink>. 1) startup error redirection. I've noticed that the main Zope service driver for Windows seems to work fine when everything is setup correctly, but when things go wrong it offers no clues as to what. This is reflected in collector item 1020 ("poor error reporting on product initialisation failure under windows"). Issue 1408 ("Configuration file imports don't see INSTANCE_HOME when running Zope as a windows service"), via the referenced thread, has evidence of someone burning a day due to this. It cost me alot of time too :) I propose: Each time the child process terminates with a non-zero return code, the tail x-bytes of the child output be written to the Windows event log, where x~2k. 2) reporting of "successful start" and "backoff" strategy. A trivial startup error (eg, PYTHONPATH not set) will cause the Zope service to hopelessly retry for a number of minutes, and not respond to shutdown requests during a retry. At the moment, as soon as the service starts it reports "successful startup" to Windows. It then begins an attempt to start the child. If the child immediately fails, the code immediately begins the "backoff" strategy. This strategy appears to have 2 main purposes: * Startup may fail due to other 'services' not having yet started, so retry in the hope they become available. * The process may die due to some obscure error - restart it. On windows, assuming we install the service to depend on the "tcpip" service, I see no reason that the first reason is valid. If the process fails quickly the first time we attempt to launch it, it is almost certainly going to fail every time we try and launch it. The current strategy also means that 3rd party services could not themselves depend on the Zope service - the Zope service will report successful startup before it really has (and therefore the dependent service may itself fail). This isn't a known requirement today, but who knows! "net start" and other front ends also fail to detect fatal errors - they all say Zope started OK. I propose: We insist the child process can be created and continues to run for x seconds (where x~5). If that fails, we report an error (never reporting to Windows that we started successfully). If the child process stays alive for this period, we report success to Windows, and then use the existing backoff strategy should it die. If the machine is heavily loaded, this 5 seconds may expire before the fatal error is hit in the child - in that case, we are simply doing what we do now - using the backoff strategy to hopelessly attempt a restart - ie, a win in most cases, and no loss in the others. 3) environment setting The service process should set a number of environment variables before spawning the child - PYTHONPATH at a minimum, and according to issue #1408, INSTANCE_HOME. It already knows these values thanks to mkzopeinstance. I'm yet to determine where these values comes from for in binary build, but I see no reason not to fix this (and possibly remove whatever magic the binary does) I propose: A few trivial os.environ insertions based on the substitutions done by mkzopeinstance, before we create the child process(es). Alternatively, we create an explicit new environment we pass to CreateProcess, but I see no good reason for that.) 4) Currently, when the process is stopped, we immediately terminate the child process. This seems dangerous. We should find a way to gracefully terminate the child, and try that before we simply kill it. I propose: That someone help me work out how to do this <wink>. I've already worked out how if the service knows the username/password of a Zope administrator, but it doesn't! Sending a Ctrl_C 'signal' doesn't work without hacks to run.py (and I'm yet to confirm it will even with such hacks). I welcome any feedback on these issues. Obviously I am willing to back each of these proposals up (except 4!) with code that seems to work :) I would also welcome feedback on the best way to proceed (ie, create a new collector for each issue? thrash it out here? give up?<wink>, etc) Note that none of these issues would require a win32all/pywin32 update. If anyone was really upset by issue 1423 ("Zope 2.7.1 won't run as service under NT"), and also able to test, I'd be willing to fix it - but that *would* require a pywin32 upgrade. Tim has already kindly filled me in on that background, so it may not be trivial (ie, I would need help!) Thanks! Mark

8 19

inconsistent mimetype assignment for uploaded files
by Jan-Wijbrand Kolman 30 Sep '04

30 Sep '04

Hello, we recently realised mimetype assignment in Zope to e.g. Zope File objects is inconsistent and can vary when different clients (browsers) upload files with the same file extensions. Example: when a file called "foobar.rtf" is upload to a Zope File object from Linux Firefox, the mimetype assigned is (can be) 'application/rtf'. However, the same file uploaded to the same Zope File object in the same Zope instance, using IE on Window2000 (with MS Office installed) will get 'application/msword' assigned. The mimetype assignment for uploaded files is done in OFS.Image.py (maybe there're more places or other Products that do this - I know that at least ExtFile does this too). line 463 of OFS.Image.py, Zope 2.7.2: def _get_content_type(self, file, body, id, content_type=None): headers=getattr(file, 'headers', None) if headers and headers.has_key('content-type'): content_type=headers['content-type'] else: if type(body) is not type(''): body=body.data content_type, enc=guess_content_type( getattr(file, 'filename',id), body, content_type) return content_type Then I understood that the headers as sent by the client for this file (may?) have a content-type entry that takes precedence over both the mimetypes 'database' and the content_type passed in as an argument. We could deal with the inconsistent assignment on the application level (in this case Silva), but I'd rather consider changing this behaviour on the Zope level. I could imagine changing the way a mimetype is 'guessed' from an uploaded File to something like: def _get_content_type(self, file, body, id, content_type=None): """ Order of precedence: 1) see if guess_content_type resolves to a mimetype for the filename 2) if not use content_type as sent in the headers if available 3) else use argument passed in """ headers = getattr(file, 'headers', {}) content_type = headers.get('content-type', content_type) if type(body) is not type(''): body = body.data name = getattr(file, 'filename', id) content_type, enc = guess_content_type(name, body, content_type) return content_type Does anyone have an opinion on this? Is the current behaviour completely intentional, maybe even according to some specification (and thus I should deal with it on the application level)? Should I file a collector issue? regards jw -- Jan-Wijbrand Kolman jw(a)infrae.com

4 5

AW: [Zope3-dev] Removing I18nFile and I18nImage from X3.0
by dev＠projekt01.ch 30 Sep '04

30 Sep '04

Stephan Richter wrote: > Betreff: Re: [Zope3-dev] Removing I18nFile and I18nImage from X3.0 > > > On Thursday 30 September 2004 06:17, Jim Fulton wrote: > > I think it would be very worth while to think about how to > > internationalize meta data. The advantage of coming up with a > > mechanism for internationalizing meta data is that it bebefits all > > content. > > I agree. Providing a good mechanism to do this would be > generally useful and > not be application specific. I have a strange feeling that > the language will > eventually become accessible via a thread global -- or can I > already get to > it via the participation? Note that if this were the case, > our translation > system would behave much more like simple applications. > > Regards, > Stephan You can remove it from the rc1. I will reimplement it in our project. Regards Roger

1 0

AW: AW: [Zope3-dev] Removing I18nFile and I18nImage from X3.0
by dev＠projekt01.ch 29 Sep '04

29 Sep '04

Stephan Richter wrote: > Gesendet: Mittwoch, 29. September 2004 21:57 > > > Uhaaa ,we need it. Ok, remove it and I will take a look at the > > IsarSprint what we can do. > > Okay. If you really use it, you should really reimplement it, > since it is > broken. ;-) Broken in the rc1? or in the trunk? Regards Roger

1 0

Zope and PythonCAD being good neighbors
by Art Haas 29 Sep '04

29 Sep '04

Hi. I'm Art Haas, the developer of PythonCAD. About one month ago I'd received a number of e-mails describing a problem between Zope and PythonCAD due to both programs using the word 'Interface' as part of their paths to their code. When using Zope people would get odd errors as the PythonCAD 'Interface' directory was searched instead of Zope's. When PythonCAD is installed, a 'PythonCAD.pth' file gets added into Python's 'site-packages' directory, modifying the sys.path variable as seen below (and reformatted for e-mail) ... $ python >>> import sys >>> sys.path ['', '/usr/lib/python23.zip', '/usr/lib/python2.3', '/usr/lib/python2.3/plat-linux2', '/usr/lib/python2.3/lib-tk', '/usr/lib/python2.3/lib-dynload', '/usr/local/lib/python2.3/site-packages', '/usr/lib/python2.3/site-packages', '/usr/lib/python2.3/site-packages/Numeric', '/usr/lib/python2.3/site-packages/PythonCAD', '/usr/lib/python2.3/site-packages/gtk-2.0'] >>> In the 'PythonCAD' directory there are two subdirectories - 'Interface' and 'Generic'. I'm preparing to make the next release of PythonCAD - the first one with initial printing support! - and I want to fix the program so that Zope and PythonCAD play well together. As I see it, one fix is to not install the 'PythonCAD.pth' file during PythonCAD's installation and adjust the various 'import' statements in the code. Another fix is to continue to have 'PythonCAD.pth' add the 'PythonCAD' directory to sys.path and add a directory below the 'PythonCAD' directory, then move 'Interface' and 'Generic' into that new directory. This choice also requires adjusting the various 'import' statements in the code, as well as some work in moving the files in Subversion. I'd like to get some suggestions for the Zope developers as for what they suggest to do. As both the Numeric and PyGTK packages adjust the sys.path variable, I'd like to keep 'PythonCAD' in there as well, though this choice probably means more work. As there is lots of Python expertise on this list, a few bits of wisdom from those in similar situations in the past would be welcomed. Thanks in advance. Art Haas P.S. I do not receive mail from the list, so please CC me in replies. I'll also check the list archives for postings. -- Man once surrendering his reason, has no remaining guard against absurdities the most monstrous, and like a ship without rudder, is the sport of every wind. -Thomas Jefferson to James Smith, 1822

2 1

session problem again?
by Matteo Memelli 29 Sep '04

29 Sep '04

Hello I include the conversation that I had today with Chris: > I'm using Zope 2.6b1+ with ZWarehouse e-commerce product > (ZWarehouse-20040522 version 0.8) > I tried upgrading to Zope 2.7 but ZWarehouse gave me many problems so > I decided to try applying your patches for Sessioning problems that I > have. Hmm.. > I think that the error that zope gives to me is very known to you... I > tried to copy the whole directories Transience, Session and ZPublisher > from both > Zope-2.7.3b1-0 and Zope 2.7.2 but without success I also copied the > file startup.py from lib/python/Zope/App/ ... > Am I missing anything? Yes, the ZODB directory also... although I reallly have no idea whether this will work under 2.6. Really, to be sure you've got all the right files, it'd be best if you could upgrade to 2.7.3... maybe we could help fix whatever in ZWarehouse doesn't work under there (move this conversation to the Zope list zope(a)zope.org?) To be thorough, you will also need to delete and recreate any persistent session data container you've got laying around (unless it's in temp_folder, that's in RAM anyway and gets recreated on reboot). > My customer is complaining so much and I really don't know what to do! > Hoping in your answer...thank you in advance > > Matteo Memelli > > > > Site Error > An error was encountered while > publishing this resource. > > KeyError > > Sorry, a site error occurred. > > Traceback (innermost last): > > * Module ZPublisher.Publish, > line 175, in > publish_module_standard > * Module Products.iHotfix, > line 83, in new_publish > * Module ZPublisher.Publish, > line 132, in publish > * Module Zope.App.startup, > line 203, in > zpublisher_exception_hook > * Module ZPublisher.Publish, > line 101, in publish > * Module ZPublisher.mapply, > line 88, in mapply > * Module ZPublisher.Publish, > line 39, in call_object > * Module OFS.DTMLDocument, > line 128, in __call__ > <DTMLDocument instance at > 41725260> > URL: > http://www.arte-luce.com/catalog/html/catalog.html/manage_main > Physical > Path:/arteluce/catalog/html/catalog.html > * Module > DocumentTemplate.DT_String, > line 474, in __call__ > * Module > DocumentTemplate.DT_Let, > line 76, in render > * Module OFS.DTMLDocument, > line 121, in __call__ > <DTMLDocument instance at > 4172b6b0> > URL: > http://www.arte-luce.com/catalog/html/zwarehouse_html_header/manage_main > Physical > Path:/arteluce/catalog/html/zwarehouse_html_header > * Module > DocumentTemplate.DT_String, > line 474, in __call__ > * Module > DocumentTemplate.DT_Util, > line 201, in eval > __traceback_info__: > cart_functions > * Module <string>, line 1, in > <expression> > * Module > Shared.DC.Scripts.Bindings, > line 306, in __call__ > * Module > Shared.DC.Scripts.Bindings, > line 343, in _bindAndExec > * Module > Products.PythonScripts.PythonScript, line 318, in _exec > * Module None, line 16, in > setSessionByRequest.py > <PythonScript at > /arteluce/catalog/cart_functions/setSessionByRequest.py> > Line 16 > * Module > ZPublisher.HTTPRequest, > line 1218, in __getattr__ > * Module > ZPublisher.HTTPRequest, > line 1178, in get > * Module > Products.Sessions.SessionDataManager, line 94, in getSessionData > * Module > Products.Sessions.SessionDataManager, line 181, in _getSessionDataObject > * Module > Products.Transience.Transience, line 847, in new_or_existing > * Module > Products.Transience.Transience, line 450, in __setitem__ > KeyError: 1096365080 (Also, an > error occurred while attempting to > render the standard error message.) > > > ___________________________________ > > Troubleshooting Suggestions > > * The URL may be incorrect. > * The parameters passed to > this resource may be > incorrect. > * A resource that this > resource relies on may be > encountering an error. > For more detailed information about > the error, please refer to error > log. > > If the error persists please > contact the site maintainer. Thank > you for your patience. > > > Hmmm.. that's an odd place to get that error. I'm surprised. Does it only happen after some period of inactivity on the site? - C FOLLOWING Chris's advice first I tried to copy also ZODB directory from Zope 2.7.3 but still the same error... so I Installed Zope-2.7.3b1-0 (IS THIS THE RIGHT VERSION THAT SHOULD SOLVE SESSION PROBLEMS?) and reinstalled with success the following packages: ZWarehouse 0.8 iHotfix-0.4.0 itools-0.4.6 Localizer-1.1.0a4 ZMySQLDA-2.0.8 MySQL-python-1.0.0 TranslationService-0.4 THEN imported my customer's web site but I still have the same problem... whats going on... what am i missing? For every change I made I delete and recreate any persistent session data container rebooting the server (IS THIS ENOUGH OR MUST I DO ANYTHING ELSE???) >From what I can see it seems that the problem comes after about 20 minutes of inactivity... session timeout? (How can I get the session longer just for now I mean??!! modyfing default value in session_timeout_minutes in lib/python/Zope/Startup/handlers.py ???) Hoping to solve the problem quickly....my customer is going to kill me otherwise :( Thank you Matte

3 2

PCNG hook into Calendar
by Bryan Simmons 29 Sep '04

29 Sep '04

I was wondering if anyone would have suggestions on the best way to have another Product create an event, an ATEvent that is, in a portal. I'm looking for an interface to the ATEvent object. I plan to modify PCNG 1.2.4 such that issues can be created as tasks that have a time-span. I'm going to use it as a scheduler for tasking my company's web hosting department. Basically, when a new issue is created, I will have it create an ATEvent if the user specifies that this issue should be a time-based task. I may end up creating a whole new Product to have this work as I would also like to have a click on an ATEvent take the browser to the PCNG issue tied to that event. Any ideas? Some times this editing of Products can be a mess and I'm just asking for any advice on how to keep it simple. -- Regards, Bryan Simmons -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. -- Rick Osborne -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2 2

trying to update an archetype field
by Sadullah Mr S 28 Sep '04

28 Sep '04

Hello, I have been trying to update an archetype field after upgrading to Plone 2.0.4 using Archetypes 1.2.5rc4. Every time i try to click on edit i get -- This message has been scanned for viruses and dangerous content by the Cranfield MailScanner, and is believed to be clean.

2 1

script to test update error
by Sadullah Mr S 28 Sep '04

28 Sep '04

Hello, I have upgraded the product plone on zope to version 2.0.4 and have been trying to edit content with the archetype product 1.2.5rc4. I get attribute error: unicode policy. Is there a simple script i could use to loop through all the content on a site and check (when in edit and then save ) the attribute error that comes up and get the path of the object where that error exists? Regards, Shayan. -- This message has been scanned for viruses and dangerous content by the Cranfield MailScanner, and is believed to be clean.

1 0

AW: AW: [Zope3-dev] Release for Views As Adapters
by dev＠projekt01.ch 28 Sep '04

28 Sep '04

Jim Fulton wrote: > Betreff: Re: AW: [Zope3-dev] Release for Views As Adapters > > > dev(a)projekt01.ch wrote: > > ... > > >>These particular changes are no cause for a new release. > > > > > > We use some removed concepts like: > > > > _context.action( > > discriminator = ('view', for_, name, > IBrowserRequest, layer), > > callable = handler, > > args = (zapi.servicenames.Presentation, 'provideAdapter', > > IBrowserRequest, new_class, name, for_, > Interface, layer, > > _context.info), > > ) > > You have your own view directive? > > Why? We can take alook at this at the IsarSprint. Or if you interessted you can see the metaconfigure at: http://svn.tiks.org/viewcvs/Tiks/trunk/src/tiks/skinbase/slots/metaconfigure .py?rev=1244&view=markup and the mixin class for the multipage directive: http://svn.tiks.org/viewcvs/Tiks/trunk/src/tiks/skinbase/slots/app/portletvi ewclass.py?rev=1236&view=markup > > Is there a way for to register this views for both version of z3? > > You might be able to call the default view directive. I > can't really tell because I don't know what you're doing. I will take a look at the new api. Thanks Roger > Jim > > -- > Jim Fulton mailto:jim@zope.com Python Powered! > CTO (540) 361-1714 http://www.python.org > Zope Corporation http://www.zope.com http://www.zope.org > > _______________________________________________ > Zope3-dev mailing list > Zope3-dev(a)zope.org > Unsub: > http://mail.zope.org/mailman/options/zope3-dev/dev%40projekt01.ch > >

2 1