Database not in sync for days ?
Hello, I had that same problem a month ago: a Zope 7.4 instance turning on a Solaris 9 with a 2.7 Gb Data.fs file, (hypo.ge.ch:8080 is for schools of secondary degrees in Geneva, I introduced zope there a few years ago and it meets a great success among teachers and schools but our base starts to grow big and we are encountering big problems to manage it with very little resources !) On saturday evening there has been a failure that required my intervention, I stopped zope and tried out the fsrecover script. Everything went OK and there had been no error found by fsrecover (?). I restarted Zope removing the Data.fs.index (I noticed that often Zope wouldn't start because of discreapencies between Data.fs and Data.fs.index => I wonder if it is not ZSync that does that mess?) It restarted OK, but all that has been made for 2 weeks was gone !!! Loking at the log to find out why, I saw in the event.log a lot of ERRORs of that kind: 2005-03-11T23:08:04 ERROR(200) ZServer uncaptured python exception, closing channel <ZServer.HTTPServer.zhttp_channel connected 213.3.93.136:189 89 at 0x77bf828 channel#: 167268 requests:> (socket.error:(11, 'Resource temporarily unavailable') [/usr/local/lib/python2.3/asynchat.py|handle_ read|88] [/prd/data/Zope/var/zope2.7.4/lib/python/ZServer/medusa/http_server.py|recv| 423] [/usr/local/lib/python2.3/asyncore.py|recv|356]) Does anyone know, what does that mean and could it be the problem I have in this configuration? Dieter Mauer told me that after each transaction zope writes into the Data.fs, how can it be that I have a base with 2 weeks of missing data from the moment I do a restart? Thank you for your help, folks, I get despaired with that problem (many teachers lose hours of their work) serge
On Mon, Mar 14, 2005 at 03:03:12PM +0100, Renfer Serge (EDU) wrote:
a Zope 7.4 instance
Which exact version? Ah, never mind, I see from your log that it's 2.7.4.
I restarted Zope removing the Data.fs.index (I noticed that often Zope wouldn't start because of discreapencies between Data.fs and Data.fs.index => I wonder if it is not ZSync that does that mess?)
Do you mean ZSyncer? If so, I doubt that it has anything to do with these discrepancies. It just uses the "normal" import / export features. I have never heard of this causing problems with Data.fs.index.
It restarted OK, but all that has been made for 2 weeks was gone !!!
Loking at the log to find out why, I saw in the event.log a lot of ERRORs of that kind:
2005-03-11T23:08:04 ERROR(200) ZServer uncaptured python exception, closing channel <ZServer.HTTPServer.zhttp_channel connected 213.3.93.136:189 89 at 0x77bf828 channel#: 167268 requests:> (socket.error:(11, 'Resource temporarily unavailable') [/usr/local/lib/python2.3/asynchat.py|handle_ read|88] [/prd/data/Zope/var/zope2.7.4/lib/python/ZServer/medusa/http_server.py|recv| 423] [/usr/local/lib/python2.3/asyncore.py|recv|356])
Does anyone know, what does that mean and could it be the problem I have in this configuration?
That means ZServer is having problems with some incoming client requests. Often these are malformed requests from script kiddies randomly scanning you for e.g. ASP vulnerabilities. The rest of Zope never even sees these requests. No effect on persistence at all. I wouldn't worry about it as far as your data problems go. But generally, it is recommended to have an "http sanitizer" in front of Zope that filters out gunk like this. Apache, Pound, whatever.
Dieter Mauer told me that after each transaction zope writes into the Data.fs, how can it be that I have a base with 2 weeks of missing data from the moment I do a restart?
It sounds likely that you have a problem with your Data.fs that is not detected by fsrecover. I would suggest taking the discussion to the zodb-dev list, they are the best resource for dealing with Filestorage disasters. Meanwhile, do you have nightly backups? If so, you may be able to recover some or all recent changes from backups (unless they are corrupt too). -- Paul Winkler http://www.slinkp.com
Renfer Serge (EDU) wrote at 2005-3-14 15:03 +0100:
... Dieter Mauer told me that after each transaction zope writes into the Data.fs, how can it be that I have a base with 2 weeks of missing data from the moment I do a restart?
I have seen this under *nix, when a process still had opened the storage file while the file was deleted and replaced. In this case, the process continues to write to the (no longer visible apart from this process) file; the new file stays unchanged. -- Dieter
participants (3)
-
Dieter Maurer -
Paul Winkler -
Renfer Serge (EDU)