ExternalMethod for exporting un-pickled Office files to the filesystem etc
Hi Just wondering if anyone has written or attempted to write a script which would read through a site built using Zope and write all attached/uploaded files (Word, PDF, Excel) etc to a filesystem location in an un-pickled format. Therefore the files would be usable by their native application and be accessible outside of Zope. The business logic behind this is to be able to create, say once a day at night, a copy of all digital business assets for extra back-up and recovery purposes in case of disaster. Also this would be good for migration purposes if required. If not, how long would it take to create such a script? I could probably source a sponsor for it. I realise that it's best to do this at the application level and at the moment this would be through Plone. I've demoed all the Plone external storage products and also checked out DirectoryStorage but all these store pickled files etc. Thanks -- michael
michael nt milne schrieb:
Hi
Just wondering if anyone has written or attempted to write a script which would read through a site built using Zope and write all attached/uploaded files (Word, PDF, Excel) etc to a filesystem location in an un-pickled format. Therefore the files would be usable by their native application and be accessible outside of Zope. The business logic behind this is to be able to create, say once a day at night, a copy of all digital business assets for extra back-up and recovery purposes in case of disaster. Also this would be good for migration purposes if required.
If not, how long would it take to create such a script? I could probably source a sponsor for it. I realise that it's best to do this at the application level and at the moment this would be through Plone.
I've demoed all the Plone external storage products and also checked out DirectoryStorage but all these store pickled files etc.
----------- snip ---------- import os import sys try: targetdir=sys.argv[1] except IndexError: print "usage: zopectl run %s <target dirrectory>" % sys.argv[0] def filedump(resulttuple,basedir): pathname,fileobject=resulttuple print "dumping %s (%d Bytes)" % (fileobject.getId(),fileobject.size) o=open(os.path.join(basedir,pathname.replace('/','_')+fileobject.getId()),"wb") o.write(str(fileobject.data)) for res in app.ZopeFind(app,obj_metatypes=['File']) filedump(res,targetdir) --------- snip ----------- and run this over ZEO via zopectl run thescript.py somewhere/to/targetdir There are possibly many improvements (creating subdirectories, fixing extensions and so on) Regards Tino Wildenhain
Cheers, thanks a lot for that. I'll give it a go. I still may be interested in sponsoring a version with the improvements you mention etc and possibly more. Would that code be compatible for Unix and also for Windows NTFS? Thanks On 6/22/06, Tino Wildenhain <tino@wildenhain.de> wrote:
michael nt milne schrieb:
Hi
Just wondering if anyone has written or attempted to write a script which would read through a site built using Zope and write all attached/uploaded files (Word, PDF, Excel) etc to a filesystem location in an un-pickled format. Therefore the files would be usable by their native application and be accessible outside of Zope. The business logic behind this is to be able to create, say once a day at night, a copy of all digital business assets for extra back-up and recovery purposes in case of disaster. Also this would be good for migration purposes if required.
If not, how long would it take to create such a script? I could probably source a sponsor for it. I realise that it's best to do this at the application level and at the moment this would be through Plone.
I've demoed all the Plone external storage products and also checked out DirectoryStorage but all these store pickled files etc.
----------- snip ---------- import os import sys
try: targetdir=sys.argv[1] except IndexError: print "usage: zopectl run %s <target dirrectory>" % sys.argv[0]
def filedump(resulttuple,basedir): pathname,fileobject=resulttuple print "dumping %s (%d Bytes)" % (fileobject.getId(),fileobject.size) o=open(os.path.join(basedir,pathname.replace ('/','_')+fileobject.getId()),"wb") o.write(str(fileobject.data))
for res in app.ZopeFind(app,obj_metatypes=['File']) filedump(res,targetdir)
--------- snip -----------
and run this over ZEO via zopectl run thescript.py somewhere/to/targetdir
There are possibly many improvements (creating subdirectories, fixing extensions and so on)
Regards Tino Wildenhain
-- michael
michael nt milne wrote:
Cheers, thanks a lot for that. I'll give it a go. I still may be interested in sponsoring a version with the improvements you mention etc and possibly more. Would that code be compatible for Unix and also for Windows NTFS?
Note: I think this deals with Files, as per Zope's built-in OFS types, not ATFile, the 'File' type you see in Plone. It may still work, but you may also need to access ATFile's API directly. Note that another version of this script may be to just use wget to pull the URLs for files. For example, you could have a script: from Products.CMFCore.utils import getToolByName catalog = getToolByName(context, 'portal_catalog') for i in catalog(portal_type = ('File', 'ATFile',)): print i.getURL() return printed ... put that in a skin folder or portal_skins custom. Then write a bash shell script or similar that first does a wget to http://my.server.com/getAllFileUrls (if that's the name of the script), saving the body of the response (a newline-delimited list of filenames) to a file or just an array in-memory (you could do this with python's urllib as well). Then, loop over the lines in that file, and do a get on each one - it should fetch the file for you. Martin -- View this message in context: http://www.nabble.com/ExternalMethod-for-exporting-un-pickled-Office-files-t... Sent from the Zope - General forum at Nabble.com.
michael nt milne schrieb:
Cheers, thanks a lot for that. I'll give it a go. I still may be interested in sponsoring a version with the improvements you mention etc and possibly more. Would that code be compatible for Unix and also for Windows NTFS?
at the moment, yes ( using os.path.join() for that) directory creating isnt that hard either, its just some recursion and checking... Regards TIno Wildenhain
participants (3)
-
Martin Aspeli -
michael nt milne -
Tino Wildenhain