[Zope] RE: URGENT: Can't start up zope

Jay, Dylan djay@lucent.com
Sat, 12 Jun 1999 15:21:23 +1000


Ok, I've solved my own problem. I removed MailHost and modified this method
in Product.py
    def _delObject(self,id):
        o = getattr(self, id)
        if hasattr(o, '_factories'):
          for factory in getattr(self, id)._factories():
factory._unregister()
        ProductFolder.inheritedAttribute('_delObject')(self, id)

This allowed me to delete the strange object from the ProductManager in
ContolPanel. Then when restarting with the MailHost product code put back
everything was fine. I'd like to thank myself for the invaluable assistance
I provided myself :)

Now for my rant. How on earth did I get in this situation???? There are two
serious problems here.
1) How on earth did an object get overridden by a completely different
object. This might have something to do with object id's getting mixed
up???? In any case its a really big bug and as I'm not sure when or how it
happened I'm not sure how we can fix it.

2) It is a seriously problem if anything in the ODB can prevent zope
starting. Since zope is only method of accessing the ODB then it can't be
fixed (easily) unless Zope can be brought back up. It seems to me that much
more care should be taken to find any unhandled exceptions in the zope init
sequence. I disliked the idea of a putting all my data into some closed
repository. I am completely left helpless if I want to rollback any changes
in situations like this. There should be an intergration into a source
control repository like CVS. Something that is file based so data can be
manually edited. 
Also is there anyway to reduce the dependence of the main zope functionality
on states kept inside the ODB? ie in this case is it necessary to  keep
persistant information on all the Products in the ODB?
In short, Zope needs to be crash proof. How can we make this happen?


> -----Original Message-----
> From:	Jay, Dylan 
> Sent:	Saturday, June 12, 1999 13:11
> To:	'zope@zope.org'
> Cc:	'support@digicool.com'
> Subject:	Re: URGENT: Can't start up zope
> 
> > -----Original Message-----
> > From: Michel Pelletier 
> > Sent: Friday, June 11, 1999 2:42 PM
> > To: 'djay@lucent.com'; zope@zope.org
> > Cc: Support
> > Subject: RE: URGENT: Can't start up zope.
> > 
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: Dylan Jay [mailto:djay@cse.unsw.edu.au]
> > > Sent: Friday, June 11, 1999 2:29 PM
> > > To: zope@zope.org
> > > Cc: support@digicool.com
> > > Subject: URGENT: Can't start up zope.
> > > 
> > > 
> > > I was doing some external method work on our production 
> > > server (silly boy)
> > > and something went very wrong. Every time I start zope it 
> > > will crash. This
> > > is both as an NT service and using ZHTTPServer. ZHTTPServer 
> > > gives a stack
> > > dump that mentions initializing products and factories and stuff
> > > (unfortunatly my office has a power outage currently so I 
> > > can't the dump
> > > or the data.bbb until 5 hours or so). Using the bbb.py it 
> reports no
> > > corruptions so I can only presume the database contains some
> > > configurations that screws up zope. To make matters worse our 
> > > expensive
> > > backup solution doesn't seem to have backed up data.bbb.
> > > 
> > 
> > Can you send us this traceback?  Did a power outage occour 
> > while the server was running?  Did the machine not get shut 
> > down properly and this this problem happened?  Or did it just 
> > happen after a clean shutdown (ie, what was the state of the 
> > server before coming up the first time with this error).
> 
> No, It occured while testing an external method.
> 
> Heres the dump
> 
> C:\Program Files\Provision>..\python\python serve.py
> Traceback (innermost last):
>   File "serve.py", line 100, in ?
>     ZopeHTTPServer.main(args)
>   File "C:\Program 
> Files\Provision\ZopeHTTPServer\ZopeHTTPServer.py", line 498,
> in main
>     start(module_file,host,port,threading,env)
>   File "C:\Program 
> Files\Provision\ZopeHTTPServer\ZopeHTTPServer.py", line 436,
> in start
>     set_published_module(module_file,BoboRequestHandler,env)
>   File "C:\Program 
> Files\Provision\ZopeHTTPServer\ZopeHTTPServer.py", line 432,
> in set_published_module
>     __import__(name) # to catch problem modules right away
>   File "C:\Program Files\Provision\lib\python\Main.py", line 103, in ?
>     Bobobase=OFS.Application.open_bobobase()
>   File "C:\Program 
> Files\Provision\lib\python\OFS\Application.py", line 371, in
> open_bobobase
>     install_products(app)
>   File "C:\Program 
> Files\Provision\lib\python\OFS\Application.py", line 486, in
> install_products
>     App.Product.initializeProduct(product, product_name, 
> package_dir, app)
>   File "C:\Program 
> Files\Provision\lib\python\App\Product.py", line 428, in init
> ializeProduct
>     if old is not None: products._delObject(name)
>   File "C:\Program 
> Files\Provision\lib\python\App\Product.py", line 145, in _del
> Object
>     for factory in getattr(self, id)._factories(): 
> factory._unregister()
> AttributeError: _factories
> 
> Now that I've had some time to debug it a little, what I 
> discover is this.
> The above crash happens while trying to initialize the 
> MailHost object. The reason is that getattr(self,id) in the 
> call above is actually a DTML object not a Factory object. 
> Somehow a piece of code got whacked in the wrong place. 
> I can now get the site up by removing the MailHost product. 
> However mail is an extermely important part of the site in 
> that no new user can register without it. When a shut it down 
> and add MailHost back in I get the same problem. What I need 
> to do is somehow take out what is posing as the MailHost 
> Product. I have no idea how to do this.  
> 
> I will resend this mail to digicool.com with my database
> 
> 
> > -Michel
> > 
> > > What I need to know are
> > > a) What the likely causes of a something like I'm describing? 
> > > I realize I
> > > havn't given much to go on, but ANY information you can give 
> > > me at this
> > > point would very very appreciated.
> > > b) What tools do I need to modify the data.bbb. I can convert 
> > > it xml ok
> > > but it seems there's no way to convert it back. Is there any 
> > > tool where I
> > > can just roll back the last few transactions? (This may not 
> > > work actually
> > > as it was probably the external method that did the damage).
> > > 
> > > In case its relevent I guess I should include some detail of 
> > > the external
> > > method I was working on. It was a method to determine if 
> a user had
> > > permission to view an object. It looked at a users roles 
> > and the roles
> > > for the view permission. It would recurse with the parent if 
> > > the object
> > > aquired roles.
> > > 
> > > Thanks in advance for any ideas anyone has.
> > > 
> > > 
> > > 
> > > Dylan.
> > > djay@lucent.com
> > > 
> >