I know zope makes a copy of Data.fs before packing. Is it possible to specify a location other than <zope>/var to place that copy? Mark
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark Gibson wrote:
I know zope makes a copy of Data.fs before packing. Is it possible to specify a location other than <zope>/var to place that copy?
Actually, it makes a *new* file when packing, and then unlinks the old file and re-links the copy under its name after finishing the pack successfully. If you are looking to "spread the love" across multiple partitions, you can symlink the original file into a directory on another drive / partition, and then do the pack there. One the pack completes, you can move or copy the packed version back to the original directory. To do the pack in the temporary directory, something like the following Python script should work:: #!/path/to/your/python import time from ZODB.FileStorage import FileStorage to_pack = FileStorage('/mnt/space/packing/Data.fs') when = time.time() - (7 * 86400) # one week ago to_pack.pack(when) Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFE9Nal+gerLs4ltQ4RAnIDAJ4uMnZFWxT2QUGIxUX8pYEU3ZAHCgCff2p5 ptA83pek4hISALzRwiX7wIU= =gz4O -----END PGP SIGNATURE-----
Tres Seaver wrote:
Mark Gibson wrote:
I know zope makes a copy of Data.fs before packing. Is it possible to specify a location other than <zope>/var to place that copy?
Actually, it makes a *new* file when packing, and then unlinks the old file and re-links the copy under its name after finishing the pack successfully. If you are looking to "spread the love" across multiple partitions, you can symlink the original file into a directory on another drive / partition, and then do the pack there. One the pack completes, you can move or copy the packed version back to the original directory.
I recently had to move a fairly large Data.fs onto a different partition, but just copying it would have involved downtime (at least for authoring) that I wanted to avoid. The solution I came up with was to create a var folder on the new partition symlinking back to Data.fs in the original location. Then I switched ZEO to using the var on the new partition and packed the database. After packing the database was on the new partition with no downtime apart from a single ZEO restart.
Duncan Booth wrote:
Tres Seaver wrote:
Mark Gibson wrote:
I know zope makes a copy of Data.fs before packing. Is it possible to specify a location other than <zope>/var to place that copy? Actually, it makes a *new* file when packing, and then unlinks the old file and re-links the copy under its name after finishing the pack successfully. If you are looking to "spread the love" across multiple partitions, you can symlink the original file into a directory on another drive / partition, and then do the pack there. One the pack completes, you can move or copy the packed version back to the original directory.
I recently had to move a fairly large Data.fs onto a different partition, but just copying it would have involved downtime (at least for authoring) that I wanted to avoid.
The solution I came up with was to create a var folder on the new partition symlinking back to Data.fs in the original location. Then I switched ZEO to using the var on the new partition and packed the database. After packing the database was on the new partition with no downtime apart from a single ZEO restart.
Nice. I should note that DirectoryStorage (http://dirstorage.sf.net) has excellent replication support which make things like moving live Zope instances from one partition to another one w/o lots of downtime almost trivial. Philipp
Philipp von Weitershausen wrote:
I should note that DirectoryStorage (http://dirstorage.sf.net) has excellent replication support which make things like moving live Zope instances from one partition to another one w/o lots of downtime almost trivial.
It's also f'ing slow and doesn't scale ;-) Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 30 Aug 2006, at 15:48, Chris Withers wrote:
Philipp von Weitershausen wrote:
I should note that DirectoryStorage (http://dirstorage.sf.net) has excellent replication support which make things like moving live Zope instances from one partition to another one w/o lots of downtime almost trivial.
It's also f'ing slow and doesn't scale ;-)
Translation: In our experience, large databases with a lot of objects can lead to very slow operations, such as packing and backing up the data. This might be improved by using a file system designed for many small files, but on ext3 the whole thing became a terrible dog. This slowness also appears when doing automated buildouts which create lots of objects. Even simple stuff like just instantiating a Plone portal from the ZMI was noticeably slower. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFE9ZyQRAx5nvEhZLIRAtDYAKCl83nHzOHCuqpfCAQ7xuY5a1nG0QCgtUG0 fPg4mYLK4ILq9e/yERDRNbs= =A5c2 -----END PGP SIGNATURE-----
Jens Vagelpohl wrote:
It's also f'ing slow and doesn't scale ;-)
Translation: In our experience, large databases with a lot of objects can lead to very slow operations, such as packing and backing up the data. This might be improved by using a file system designed for many small files, but on ext3 the whole thing became a terrible dog. This slowness also appears when doing automated buildouts which create lots of objects. Even simple stuff like just instantiating a Plone portal from the ZMI was noticeably slower.
I thought my version was much more understandable ;-) Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (6)
-
Chris Withers -
Duncan Booth -
Jens Vagelpohl -
Mark Gibson -
Philipp von Weitershausen -
Tres Seaver