GADFLY Database: Can it lose data if the Zope process is aborted?
Hi! We have a very strange case of data loss: A client of us is using a Gadfly DB to store some news items. He reported that he lost a couple of his most recent data entries when we restarted our servers after applying the latest Hotfix. I know that Gadfly is not to be used for serious applications etc., but I thought that this was because of performance issues, not because it can lose data. Has anybody had similar experiences? Our first tests say that Gadfly stores all INSERTs into the DB file right after they have been commited. So the only explanation for data loss would be some strange caching that is not always used ... Joachim
I thought gadfly didn't store data between restarts. Or am I losing it? Cheers. -- Andy McKay. ----- Original Message ----- From: "Joachim Werner" <joe@iuveno-net.de> To: <zope-dev@zope.org> Sent: Thursday, July 26, 2001 11:17 AM Subject: [Zope-dev] GADFLY Database: Can it lose data if the Zope process is aborted?
Hi!
We have a very strange case of data loss: A client of us is using a Gadfly DB to store some news items. He reported that he lost a couple of his most recent data entries when we restarted our servers after applying the latest Hotfix.
I know that Gadfly is not to be used for serious applications etc., but I thought that this was because of performance issues, not because it can lose data. Has anybody had similar experiences? Our first tests say that Gadfly stores all INSERTs into the DB file right after they have been commited. So the only explanation for data loss would be some strange caching that is not always used ...
Joachim
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
I thought gadfly didn't store data between restarts. Or am I losing it?
In most cases it does. It just holds tables in memory for querying. But it DOES store to the filesystem. It can be checked easily as it uses a very ASCII-like format and all the new records can be seen in the DB files.
I know that Gadfly is not to be used for serious applications etc., but I thought that this was because of performance issues, not because it can lose data. Has anybody had similar experiences? Our first tests say that Gadfly stores all INSERTs into the DB file right after they have been commited. So the only explanation for data loss would be some strange caching that is not always used ...
I don't know about the guts of Gadfly, but you may have run afoul of Python's file handling. The docs warn that data written to a file object can not be counted on to appear on the disk until it is either closed or flushed. (I think, though, that this is very rarely a problem when dealing with human timespans.) I would think that an orderly shutdown of Zope would instruct Gadfly to shutdown gracefully, but I can't be sure of that. I've never had any persistence problems with Gadfly, but then, I've never done anything really substantial with it. --jcc (bottlefly)
I don't know about the guts of Gadfly, but you may have run afoul of Python's file handling. The docs warn that data written to a file object can not be counted on to appear on the disk until it is either closed or flushed. (I think, though, that this is very rarely a problem when dealing with human timespans.) I would think that an orderly shutdown of Zope would instruct Gadfly to shutdown gracefully, but I can't be sure of that.
O.k., the time span was about a day, so normally that should mean that everything has been flushed. But who knows? I have read something about a Gadfly option that does long-time caching instead of frequent commits to the file system. Whether that one is active in Zope's version of Gadfly I don't know. Zope definitely does not shutdown "gracefully". Even if I use the official "stop" skript it just kills the process ("kill" as in "killer"). That means that the sub-processes do not get a termination message or so. They are just killed. The "stop" button may be a bit more graceful, but I haven't checked that yet. Joachim
participants (3)
-
Andy McKay -
J. Cameron Cooper -
Joachim Werner