converting Python ap to Zope
Hi ZoPeople, I have searched and read everything I could find for weeks, I still need help. I am attempting to convert a Python application into Zope. I am having much trouble creating a Product from this application because of some of its dependances. The user interface is completely separate, so that is not an issue. I have tried loading the whole app as an external method, which didn't work, but I learned to make a copy of ALL modules and place them in a Zope sub-folder. The python path from my stand-alone python 2.1 didn't get searched. I am attempting to write a Product to access my application, but have run into several errors mostly due to pickleing. I would like to have the same code for my base classes, but I have the following conundrums. Any pointers would be much appreciated. 1) There are a lot of classes and tables. Each user can create multiple sets of the application classes and tables. Each set of instances and attributes is pickled as a file. The application is basically just a wrapper for the pickled instances. I can think of no way to convert the pickle file into classes and attributes that Zope can see and/or modify. It seems I will have to re-factor each and every python class as a Zope class. 2) several specialized data files are involved, which are opened and positioned based on the classes and parameters. The files are needed for all aspects of the application. Since Zope cannot pickle open file handles, I could store file names, positions, and other attributes, but I don't know how to re-open the files any time Zope restarts so that the user just references my object name (and methods) to access the data. 3) the key to accessing these specialized files is a windows .DLL file that works great in Python, but is unpickleable (is that a word?) in Zope. I think I want to create a function that gets loaded when the Product does, but I cannot find any examples of Products that do this: allow instanciated objects to call custom functions created from __init__.py in a Product. There must be someplace to add globally available functions to Zope outside of ZODB, yes? --Jerry -- /\_/\ Jerry McRae \|||/ mail safely: The Bat! 1.53d (~o o~) (stuck in win98) (. .) Zope v2.4.1 - dryer than golf )'Y'( _______________-o00-(_)-00o-____________________________________ ( ) / Imagination rules the world. --Napoleon
Jerry McRae wrote:
Hi ZoPeople,
I have searched and read everything I could find for weeks, I still need help.
I am attempting to convert a Python application into Zope. I am having much trouble creating a Product from this application because of some of its dependances. The user interface is completely separate, so that is not an issue.
I have tried loading the whole app as an external method, which didn't work, but I learned to make a copy of ALL modules and place them in a Zope sub-folder. The python path from my stand-alone python 2.1 didn't get searched.
I am attempting to write a Product to access my application, but have run into several errors mostly due to pickleing.
I would like to have the same code for my base classes, but I have the following conundrums. Any pointers would be much appreciated.
1) There are a lot of classes and tables. Each user can create multiple sets of the application classes and tables. Each set of instances and attributes is pickled as a file. The application is basically just a wrapper for the pickled instances. I can think of no way to convert the pickle file into classes and attributes that Zope can see and/or modify. It seems I will have to re-factor each and every python class as a Zope class.
2) several specialized data files are involved, which are opened and positioned based on the classes and parameters. The files are needed for all aspects of the application. Since Zope cannot pickle open file handles, I could store file names, positions, and other attributes, but I don't know how to re-open the files any time Zope restarts so that the user just references my object name (and methods) to access the data.
3) the key to accessing these specialized files is a windows .DLL file that works great in Python, but is unpickleable (is that a word?) in Zope. I think I want to create a function that gets loaded when the Product does, but I cannot find any examples of Products that do this: allow instanciated objects to call custom functions created from __init__.py in a Product. There must be someplace to add globally available functions to Zope outside of ZODB, yes?
--Jerry
-- /\_/\ Jerry McRae \|||/ mail safely: The Bat! 1.53d (~o o~) (stuck in win98) (. .) Zope v2.4.1 - dryer than golf )'Y'( _______________-o00-(_)-00o-____________________________________ ( ) / Imagination rules the world. --Napoleon
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
when you place your non zope stuff into a subdir in the products folder you can import it via: from Products.<yourProductFolder>.<Module> import <ClassName> if something is not pickleable you have to avoid that you don't asign it via "self." to a zope object instance, because zope tries to pickle it to zopedb. i got the same problem with a db connection in a module. another way could be to disable persistence for the particular python product - but i never tried this till now. hope this helps, bernd dorn
Hi! I am a bit confused by your problems. Let's see what I can comment on ...
I have tried loading the whole app as an external method, which didn't work, but I learned to make a copy of ALL modules and place them in a Zope sub-folder. The python path from my stand-alone python 2.1 didn't get searched.
I'd definitely go with a Zope Product, not an External Method ... If you configure your Zope right and actually make it use your standalone Python instead of its own one, all the stuff in your Python path is fully available. E.g., if you have installed PIL (Python Imaging Library), it is available from your Zope Products via "import PIL" as usual.
1) There are a lot of classes and tables. Each user can create multiple sets of the application classes and tables. Each set of instances and attributes is pickled as a file. The application is basically just a wrapper for the pickled instances. I can think of no way to convert the pickle file into classes and attributes that Zope can see and/or modify. It seems I will have to re-factor each and every python class as a Zope class.
This irritates me most. If you have the ZODB, why would you want to do pickling on your own? I guess some refactoring of the code and doing it the Zope way would help a lot here. Even if you used Python standalone, using the ZODB as a storage would be much more comfortable than pickling to files. Basically, to move from a "normal" class to a persistent class, you'd just have to import a few base classes (mainly "Persistent"). You won't have to convert the data structures. Just rewrite the code a bit. Look at http://www.zope.org/Documentation/ZDG/Persistence.stx for the details, if you haven't already. Another question: Are you talking about pickling instances of classes (which is trivial in Zope) or classes themselves (which is currently impossible)? In Zope, classes (except for ZClasses) are filesystem-based code, and instances are in the ZODB (though instances are classes, too, if we want to be syntactically correct).
2) several specialized data files are involved, which are opened and positioned based on the classes and parameters. The files are needed for all aspects of the application. Since Zope cannot pickle open file handles, I could store file names, positions, and other attributes, but I don't know how to re-open the files any time Zope restarts so that the user just references my object name (and methods) to access the data.
3) the key to accessing these specialized files is a windows .DLL file that works great in Python, but is unpickleable (is that a word?) in Zope. I think I want to create a function that gets loaded when the Product does, but I cannot find any examples of Products that do this: allow instanciated objects to call custom functions created from __init__.py in a Product. There must be someplace to add globally available functions to Zope outside of ZODB, yes?
ALL the functions and methods defined in filesystem-based class definitions are outside the ZODB. Let's take an example: If you have a user folder implementation in Zope, the ZODB will only store the user info. All the actual code, like the password validation, encryption etc. is outside the ZODB of course. Why would you want to pickle a .DLL file? Or did I get that wrong? Your problem sounds a bit as if you wanted to store a full version of Word 2000 with every .doc file you create because you might need it the next time you work with the document. But maybe I just don't get the point ... Make sure that you really understand the concept of Persistence before you go on. It is mainly about programming code as if it was running forever (i.e. stateful), but actually Zope can store the state between the (stateless) web requests. Cheers Joachim
Hi Joachim, - Thanks for your response. Apologize for the obtuseness. I [attempt to] clarify some of my questions herein. re: what you <JW> wrote Jerry on Saturday, December 01, 2001 at 7:50 AM: JW> If you configure your Zope right and actually make it use your standalone JW> Python instead of its own one, all the stuff in your Python path is fully JW> available. E.g., if you have installed PIL (Python Imaging Library), it is JW> available from your Zope Products via "import PIL" as usual. I must have thought that would mess with Zope. Works fine to replace the path to Zope's Python with just 'python' in the START.BAT. I do get an error when Zope starts now (my localhost version in win98): 2001-12-03T19:24:47 PROBLEM(100) Init Ambiguous name for method of Products.Site Access.SiteRoot.SiteRoot: "manage" != "manage_main" I was not getting this when I used the python from the zope directories. I have the SiteAccess Product installed, but cannot find any SiteRoot or VHM objects used anywhere. Question 1: is this important? What do I do about it? -- JW> This irritates me most. If you have the ZODB, why would you want to do JW> pickling on your own? I guess some refactoring of the code and doing it the JW> Zope way would help a lot here. Even if you used Python standalone, using JW> the ZODB as a storage would be much more comfortable than pickling to files. I *don't* want to pickle on my own. That is just the way the application is currently saving the tables. I would like to un-pickle the existing files using the modified ZODB persistent classes but since pickle loads the classes from the original modules, I don't see how this is possible. (Q2: is it?) I suppose I could write a conversion program copies each class (and attributes) and data structure manually over to a Zope instance. JW> Basically, to move from a "normal" class to a persistent class, you'd just JW> have to import a few base classes (mainly "Persistent"). You won't have to JW> convert the data structures. Just rewrite the code a bit. Look at JW> http://www.zope.org/Documentation/ZDG/Persistence.stx for the details, if JW> you haven't already. I have read that chapter. Part of the problem is that I have read *everything* I could find, and I think my brain is full. ;) I will write wrapper classes for ZODB with the same names, so that I can use the same code, and see if that will work for me. JW> Why would you want to pickle a .DLL file? Or did I get that wrong? Your JW> problem sounds a bit as if you wanted to store a full version of Word 2000 JW> with every .doc file you create because you might need it the next time you JW> work with the document. But maybe I just don't get the point ... I don't want to pickle the .dll. I just want to be able to call the .dll from anywhere in Zope. I was thinking something like the following (invalid) code in my __init__ for the Product: import Mk4py root['mk'] = Mk4py So all methods could access code like "mk.open(...)" using the same object. I might be thinking down the wrong path. (I have been know to do that on occasion). My app requires speed and flexibility. 20-200k records. add/remove fields quickly on request. some fields need to be nested and/or multi-valued. Chose MetaKit. MetaKit requires only one .dll to operate. I need two things I have yet done successfully: 1) load a class or function into Zope that imports this dll that I can call (Q3), and 2) once an object is set up, I need the file pointers not pickled (using _v_ attributes I guess), *and* the files & tables pointers re-opened if Zope re-starts, so the user doesn't have to do it manually (Q3). please lead me to [in]sight! -- /\_/\ Jerry McRae \|||/ mail safely: The Bat! 1.53d (~o o~) (stuck in win98) (. .) Zope v2.4.1 - dryer than golf )'Y'( _______________-o00-(_)-00o-____________________________________ ( ) / I used to be undecided, now I'm not sure! ========== your original message below ========== From: Joachim Werner To: Jerry McRae on: 7:50:20 AM re: [Zope] converting Python ap to Zope wise words from Joachim: JW> Hi! JW> I am a bit confused by your problems. Let's see what I can comment on ...
I have tried loading the whole app as an external method, which didn't work, but I learned to make a copy of ALL modules and place them in a Zope sub-folder. The python path from my stand-alone python 2.1 didn't get searched.
JW> I'd definitely go with a Zope Product, not an External Method ... JW> If you configure your Zope right and actually make it use your standalone JW> Python instead of its own one, all the stuff in your Python path is fully JW> available. E.g., if you have installed PIL (Python Imaging Library), it is JW> available from your Zope Products via "import PIL" as usual.
1) There are a lot of classes and tables. Each user can create multiple sets of the application classes and tables. Each set of instances and attributes is pickled as a file. The application is basically just a wrapper for the pickled instances. I can think of no way to convert the pickle file into classes and attributes that Zope can see and/or modify. It seems I will have to re-factor each and every python class as a Zope class.
JW> This irritates me most. If you have the ZODB, why would you want to do JW> pickling on your own? I guess some refactoring of the code and doing it the JW> Zope way would help a lot here. Even if you used Python standalone, using JW> the ZODB as a storage would be much more comfortable than pickling to files. JW> Basically, to move from a "normal" class to a persistent class, you'd just JW> have to import a few base classes (mainly "Persistent"). You won't have to JW> convert the data structures. Just rewrite the code a bit. Look at JW> http://www.zope.org/Documentation/ZDG/Persistence.stx for the details, if JW> you haven't already. JW> Another question: Are you talking about pickling instances of classes (which JW> is trivial in Zope) or classes themselves (which is currently impossible)? JW> In Zope, classes (except for ZClasses) are filesystem-based code, and JW> instances are in the ZODB (though instances are classes, too, if we want to JW> be syntactically correct).
2) several specialized data files are involved, which are opened and positioned based on the classes and parameters. The files are needed for all aspects of the application. Since Zope cannot pickle open file handles, I could store file names, positions, and other attributes, but I don't know how to re-open the files any time Zope restarts so that the user just references my object name (and methods) to access the data.
3) the key to accessing these specialized files is a windows .DLL file that works great in Python, but is unpickleable (is that a word?) in Zope. I think I want to create a function that gets loaded when the Product does, but I cannot find any examples of Products that do this: allow instanciated objects to call custom functions created from JW> __init__.py in a Product. There must be someplace to add globally available functions to Zope outside of ZODB, yes?
JW> ALL the functions and methods defined in filesystem-based class definitions JW> are outside the ZODB. Let's take an example: If you have a user folder JW> implementation in Zope, the ZODB will only store the user info. All the JW> actual code, like the password validation, encryption etc. is outside the JW> ZODB of course. JW> Why would you want to pickle a .DLL file? Or did I get that wrong? Your JW> problem sounds a bit as if you wanted to store a full version of Word 2000 JW> with every .doc file you create because you might need it the next time you JW> work with the document. But maybe I just don't get the point ... JW> Make sure that you really understand the concept of Persistence before you JW> go on. It is mainly about programming code as if it was running forever JW> (i.e. stateful), but actually Zope can store the state between the JW> (stateless) web requests. JW> Cheers JW> Joachim
Hi Joachim, - re: what you <JW> wrote Jerry on Saturday, December 01, 2001 at 7:50 AM: one more thing. JW> http://www.zope.org/Documentation/ZDG/Persistence.stx for the details, if I tried these demos for using ZODB for persistence. Worked great. But when I tried to open live data.fs there are two obvious problems. 2) the structure of data.fs is *complex*, and I didn't know where to start with putting data in that would be accessible by Zope users, and 1) it cannot be accessed while Zope is running. I have not seen any examples or comments about Products that use additional ZODB files so they can be accessed (when not in use) while Zope is running. Has anyone done this? Is there something "bad" about using another ZODB for occasional use inside of Zope? Thanks again. -- /\_/\ Jerry McRae \|||/ mail safely: The Bat! 1.53d (~o o~) (stuck in win98) (. .) Zope v2.4.1 - dryer than golf )'Y'( _______________-o00-(_)-00o-____________________________________ ( ) / History repeats itself, and that's one of the things that's wrong with history. --Clarence Darrow
Hi! To be honest, your app seems to be more complex than I can bear at the moment. So again only some minor hints ...
I tried these demos for using ZODB for persistence. Worked great. But when I tried to open live data.fs there are two obvious problems. 2) the structure of data.fs is *complex*, and I didn't know where to start with putting data in that would be accessible by Zope users, and 1) it cannot be accessed while Zope is running.
Use ZEO for that. You can export a ZODB via ZEO and then use it from different clients (be it a Zope or something else). Never done that, but there is some howto or so about it ... The other mail: 2001-12-03T19:24:47 PROBLEM(100) Init Ambiguous name for method of Products.Site Access.SiteRoot.SiteRoot: "manage" != "manage_main" This is nothing bad. But it SHOULD be the same whether you use your own Python or Zope's. Very funny ... JW> This irritates me most. If you have the ZODB, why would you want to do JW> pickling on your own? I guess some refactoring of the code and doing it the JW> Zope way would help a lot here. Even if you used Python standalone, using JW> the ZODB as a storage would be much more comfortable than pickling to files. "I *don't* want to pickle on my own. That is just the way the application is currently saving the tables. I would like to un-pickle the existing files using the modified ZODB persistent classes but since pickle loads the classes from the original modules, I don't see how this is possible. (Q2: is it?) I suppose I could write a conversion program copies each class (and attributes) and data structure manually over to a Zope instance." Everything is possible. But I'd still prefer a more radical refactoring, where you change the storage implementation to ZODB. Your ORIGINAL classes become Zope base classes, so there is no problem with having two base classes for the same pickle or so. JW> Why would you want to pickle a .DLL file? Or did I get that wrong? Your JW> problem sounds a bit as if you wanted to store a full version of Word 2000 JW> with every .doc file you create because you might need it the next time you JW> work with the document. But maybe I just don't get the point ... "I don't want to pickle the .dll. I just want to be able to call the .dll from anywhere in Zope. I was thinking something like the following (invalid) code in my __init__ for the Product: import Mk4py root['mk'] = Mk4py So all methods could access code like "mk.open(...)" using the same object. I might be thinking down the wrong path. (I have been know to do that on occasion)." Do it as you'd do it from Python. There is nothing Zope-specific about it ... Joachim
[Jerry McRae]
I am attempting to convert a Python application into Zope. I am having much trouble creating a Product from this application because of some of its dependances. The user interface is completely separate, so that is not an issue.
I have tried loading the whole app as an external method, which didn't work, but I learned to make a copy of ALL modules and place them in a Zope sub-folder. The python path from my stand-alone python 2.1 didn't get searched.
You can handle this by creating a .pth file and putting it in the top-level directory of Zope's python installation. Look for a post by me on this subject about a week ago, or just read the Python docs to find out about .pth files. That will take care of this part of your problems. Given the ideosycratic nature of your code, I'd seriously consider running it as a set of External Methods. That way you only have to write the interface, not rewrite the classes and tables. Make sure to keep the code out of the Extensions folder - that folder should only contain little methods that dispatch to the actual code. Cheers, Tom P
participants (5)
-
Jerry McRae -
Jerry McRae -
Joachim Werner -
Thomas B. Passin -
zope-mailinglist