[Zope-dev] Help! Need to extract files out of Data.fs!

[Dennis Allison]

One more thing, what does the disk where the Data.fs resides look like?  


On Mon, 28 Aug 2006, Dennis Allison wrote:

> 
> Kris, 
> 
> I assume you have a backup of the Data.fs.  If you do, nothing is really
> lost.
> 
> Have you compressed the Data.fs?  There is an option in the control panel
> to do this.  I can be run on a live machine, but you may not want to do so
> given the size of stuff.  I'd do it on a quiet machine.  This will 
> remove any cruft you have in the Data.fs and will improve your speed of 
> access somewhat,  Things are probably slow because of swapping.  Reducing 
> the size of Data.fs may speed things up.
> 
> Take a look at the documentation for the ZODB (Google is your friend).  
> You could write a program in Python to simply open the ZODB and extract
> the files you need.  It might save a bit of overhead.
> 
> Remember 1.3TB is a large file and is going to take some time to process 
> no matter what.   
> 
> 
> On Mon, 28 Aug 2006, Kris Adcock wrote:
> 
> > Hiya,
> > 
> > My company has been using a Zope server to store some reference material, but
> > it has grown beyond belief, and I need to extract all the collected files
> > (about 1.3 terabytes!) from the Data.fs into distinct files onto a different
> > server.
> > 
> > Unfortunately, I've run into some problems. Firstly, the FTP transfer process
> > seems to be extremely slow, and if I try and transfer too large or deep a
> > folder structure in one go then things grind to a halt.
> > 
> > To beat this I've been carefully recreating the folder structure manually,
> > and copying things in easier-to-digest chunks. Unfortunately, because I was a
> > bit rushed over the weekend I have accidentally deleted about 130,000 files!
> > 
> > I've tried going through the "Undo" procedure, but the idea of manually
> > ticking 130,000 boxes (in pages of 20) and undoing them fills me with dread.
> > So ...
> > 
> > Please please /please/ .... does anyone happen to have the file format for
> > the Data.fs file so that I could write a C program to extract and recreate
> > the files and folders to another server? Or has anyone written such a program
> > already? If so, I'll happily have your children if you can help me out! As it
> > stands, I'm going to have to break it to the librarian (the poor man who has
> > been collating all the reference material for the last six months) that I've
> > lost half of his work, and blaming it on the poor FTP client in Windows seems
> > a bit lame!
> > 
> > If it helps: I'm running Zope 2.9.2 on a SuSE Linux box. I have full access
> > to the Linux box ... and a large hammer, if I need it! :(
> > 
> > Help!
> > 
> > Many thanks in advance,
> > 
> > Kris (considering giving up this computing job and going to live in a cave
> > somewhere).
> > _______________________________________________
> > Zope-Dev maillist  -  Zope-Dev at zope.org
> > http://mail.zope.org/mailman/listinfo/zope-dev
> > **  No cross posts or HTML encoding!  **
> > (Related lists - 
> >  http://mail.zope.org/mailman/listinfo/zope-announce
> >  http://mail.zope.org/mailman/listinfo/zope )
> > 
> 
> 

--