accelerating Zope restart
Hi, My Data.fs is huge (21GB) because it contains many video files. This is not a problem in itself, however, I recently had to reboot my machine and now zope takes around 20-30 minutes to restart. Is a fstest automatically done ? How could I accelerate the restarting, if possible ? Thanks a lot for any advice, Eric
--On 31. August 2006 12:18:24 +0100 egf05@doc.ic.ac.uk wrote:
Hi,
My Data.fs is huge (21GB) because it contains many video files. This is not a problem in itself, however, I recently had to reboot my machine and now zope takes around 20-30 minutes to restart. Is a fstest automatically done ? How could I accelerate the restarting, if possible ?
Zope does maintain a Data.fs.index file. If it becomes corrupt e.g. through an unclean shutdown then Zope will recreate it by scanning the whole Data.fs file when starting Zope the next time. This will take some time....so before rebooting you should shutdown down your Zope properly. -aj -- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: info@zopyx.com - Phone +49 - 7071 - 793376 E-Publishing, Python, Zope & Plone development, Consulting
Zope does maintain a Data.fs.index file. If it becomes corrupt e.g. through an unclean shutdown then Zope will recreate it by scanning the whole Data.fs file when starting Zope the next time. This will take some time....so before rebooting you should shutdown down your Zope properly.
-aj
OK, thanks. Eric
Zope does maintain a Data.fs.index file. If it becomes corrupt e.g. through an unclean shutdown then Zope will recreate it by scanning the whole Data.fs file when starting Zope the next time. This will take some time....so before rebooting you should shutdown down your Zope properly.
OK, I have two questions about shutting down zope: - To cleanly shut down zope daemon, is "zopectl stop" enough ? - If zope runs in the foreground, how to shut it down cleanly ? CTRL+C won't do it I guess ? Eric
On Thu, 2006-31-08 at 12:52 +0100, egf05@doc.ic.ac.uk wrote:
Zope does maintain a Data.fs.index file. If it becomes corrupt e.g. through an unclean shutdown then Zope will recreate it by scanning the whole Data.fs file when starting Zope the next time. This will take some time....so before rebooting you should shutdown down your Zope properly.
OK, I have two questions about shutting down zope:
- To cleanly shut down zope daemon, is "zopectl stop" enough ?
Yes.
- If zope runs in the foreground, how to shut it down cleanly ? CTRL+C won't do it I guess ?
I always thought CTRL+C would do it actually. It's only a force kill with -9 that will kill it without shutting down properly. But you could always go to the Control_Panel in the zmi and do a shutdown from there, that is for certain safe. - Rocky -- Rocky Burt ServerZen Software -- http://www.serverzen.com News About The Server (blog) -- http://www.serverzen.net
egf05@doc.ic.ac.uk wrote:
My Data.fs is huge (21GB) because it contains many video files. This is not a problem in itself, however, I recently had to reboot my machine and now zope takes around 20-30 minutes to restart. Is a fstest automatically done ? How could I accelerate the restarting, if possible ?
Use ZEO. Make sure the storage server shuts down properly so it saves its indexes to disk. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Thu, Aug 31, 2006 at 12:23:06PM +0100, Chris Withers wrote:
egf05@doc.ic.ac.uk wrote:
My Data.fs is huge (21GB) because it contains many video files. This is not a problem in itself, however, I recently had to reboot my machine and now zope takes around 20-30 minutes to restart. Is a fstest automatically done ? How could I accelerate the restarting, if possible ?
Use ZEO.
I can see how that could let Zope restart faster. But ZEO can have *severe* performance implications for large blobs such as videos. See the bottom of http://www.slinkp.com/code/zopestuff/blobnotes Maybe worth noting that DirectoryStorage starts up fast regardless of storage size. http://dirstorage.sourceforge.net/ It also does not require RAM proportional to storage size. It is also very reliable and has some other nice features. The downsides: - You need a filesystem that can handle very large number of tiny files. - Packing is VERY slow (21 GB could easily take all night). - Somewhat slower than Filestorage in general. - Requires more disk space (+30%). This page is old but maybe still useful: http://cvs.zope.org/ZODB3/Doc/storages.html?rev=1 -- Paul Winkler http://www.slinkp.com
----- Original Message ----- From: <egf05@doc.ic.ac.uk> To: <zope@zope.org> Sent: Thursday, August 31, 2006 7:18 AM Subject: [Zope] accelerating Zope restart
Hi,
My Data.fs is huge (21GB) because it contains many video files. This is not a problem in itself, however, I recently had to reboot my machine and now zope takes around 20-30 minutes to restart. Is a fstest automatically done ? How could I accelerate the restarting, if possible ?
You may also want to have a look at the localfs product which would allow you to store your large video files on the local file system instead of inside the ZODB (but still access them as if they were stored in the ZODB). http://sourceforge.net/projects/localfs Jonathan
On Thu, 2006-31-08 at 12:18 +0100, egf05@doc.ic.ac.uk wrote:
Hi,
My Data.fs is huge (21GB) because it contains many video files. This is not a problem in itself, however, I recently had to reboot my machine and now zope takes around 20-30 minutes to restart. Is a fstest automatically done ? How could I accelerate the restarting, if possible ?
I highly recommend you use some sort of external storage product to store such large files. Perhaps Chris McDonough's blob product might fit well here. - Rocky -- Rocky Burt ServerZen Software -- http://www.serverzen.com News About The Server (blog) -- http://www.serverzen.net
participants (6)
-
Andreas Jung -
Chris Withers -
egf05＠doc.ic.ac.uk -
Jonathan -
Paul Winkler -
Rocky Burt