June 2003 - Zope - Zope lists

Backing up Data.fs
by jamesd＠mena.org.au 03 Jun '03

03 Jun '03

I'm currently trying to work out an effective backup solution for my zope servers. My current system involves shutting down the server, copying the zodb, and then starting the server again. This works, but leaves the site down for several minutes. Is it possible to do the equivilant of a sync and remount read-only? Or is it safe to just copy the ZODB from a live site? I'd also like to hear ideas on how others have setup their backup systems. exporting is not an option as my plone sites do not always correctly reimport. -James Davies (jamesd(a)mena.org.au)

6 8

Zope-2.5.0 stopped starting up
by Daniel Mahler 03 Jun '03

03 Jun '03

Hi, A Zope server that has been serving us faithfully for many months has stopped restarting. When it was brought down, Data.fs was hitting the Linux 2G file limit, but Zope was still operating correctly, but was refusing zwiki updates. I brought it down so I could back up Data.fs in a consistent state. When I try to bring it back up I get a continuous stream of error messages: /home/mahler/local/lib/python2.1/regsub.py:15: DeprecationWarning: the regsub module is deprecated; please use re.sub() DeprecationWarning) !k/usr/local/zope/lib/python/ts_regex.py:15: DeprecationWarning: the regex module is deprecated; please use the re module import regex, regsub #, Sync /home/mahler/local/lib/python2.1/regsub.py:15: DeprecationWarning: the regsub module is deprecated; please use re.sub() DeprecationWarning) /usr/local/zope/lib/python/ts_regex.py:15: DeprecationWarning: the regex module is deprecated; please use the re module import regex, regsub #, Sync /home/mahler/local/lib/python2.1/regsub.py:15: DeprecationWarning: the regsub module is deprecated; please use re.sub() DeprecationWarning) /usr/local/zope/lib/python/ts_regex.py:15: DeprecationWarning: the regex module is deprecated; please use the re module import regex, regsub #, Sync /home/mahler/local/lib/python2.1/regsub.py:15: DeprecationWarning: the regsub module is deprecated; please use re.sub() DeprecationWarning) /usr/local/zope/lib/python/ts_regex.py:15: DeprecationWarning: the regex module is deprecated; please use the re module import regex, regsub #, Sync /home/mahler/local/lib/python2.1/regsub.py:15: DeprecationWarning: the regsub module is deprecated; please use re.sub() DeprecationWarning) /usr/local/zope/lib/python/ts_regex.py:15: DeprecationWarning: the regex module is deprecated; please use the re module import regex, regsub #, Sync /home/mahler/local/lib/python2.1/regsub.py:15: DeprecationWarning: the regsub module is deprecated; please use re.sub() DeprecationWarning) As far as I know nothing was done to the installation, and it used to work just fine. Any ideas ?!? thanks D

6 9

Call for Participants for Zope 3 Sprint before Europython 2003 in Belgium (correction)
by Grégoire Weber 03 Jun '03

03 Jun '03

Hi everybody, As Guido van Rossum has noticed june 24 is a tuesday not a thursday :-) To avoid confusion below the corrected version of the Zope 3 sprint call for participants. -------------------- The Zope 3 sprint in Louvain-la-Neuve at the local university before the Europython conference is now definitive. When: Three days from june 22 to 24, 2003 (sunday to tuesday). Registration: If you decided to participate the sprint just drop me a e-mail and register your name and e-mail in the sprint wiki (see below). Accomodation: Take care to decide on participation as early as possible. Quick reservation is needed because there seems to be only little place to stay in Louvain-la-Neuve! If possible, plan to arrive on saturday evening/night. This way we don't lose time on sunday morning. Godefroid will help us moving from Louvain-la-Neuve to Charleroi Wednesday morning. So it would make sense to book the night in Louvain-la-Neuve from tuesday to wednesday also. Sprint Coaches and Topics: Experienced people like Steve Alexander, the codeworks people (Albertas Agejevas, Marius Gedminas, Viktorija Zaksiene), Martijn Faassen and possibly Godefroid Chapelle will act as Coaches. Topics will be determined definitively during the next weeks. Sprint Preparation: There is a good description about how to prepare for a sprint on <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/CMSprintPr…> Further information: <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/EuroPython…> More infos to come. Questions? gregoire.weber at switzerland.org regards, Godefroid Chapelle and Gregoire Weber _____________________________________ Grégoire Weber Rigistr. 31 CH-8006 Zürich Switzerland phone: +41-(0)1-361 66 11 mobile: +41-(0)79-44 11 457 mailto:gregoire.weber@switzerland.org

1 0

RE: [Zope] Persistence - I know I'm not doing this right...
by Samir Mishra 03 Jun '03

03 Jun '03

My question was a simplification of what I need to do. I need to store arrays of dictionaries, with 200 or more elements. And I need to access these from multiple points. In this context, can I still set variables in documents or other objects? Thanks. -----Original Message----- From: Sumiya Sakoda [mailto:sakoda@toyoake.or.jp] Sent: Tuesday, June 03, 2003 11:20 To: Samir Mishra Cc: Zope Mailing List Subject: Re: [Zope] Persistence - I know I'm not doing this right... At Tue, 3 Jun 2003 10:35:04 +0400, Samir Mishra wrote: > When I want to access 'x' & 'y', I call the above script. > > Q: What's the right way to store persistent objects/data within Zope? set the variables in object's property (for example DTML Document) may be light way (not right way) :-) - Sumiya Sakoda <sakoda(a)toyoake.or.jp>

3 2

APACHE SSL AND ZOPE....
by Bill 03 Jun '03

03 Jun '03

Basically, I am trying to stick Zope behind n APACHE Server/mod_ssl configuration. Following the instructions for Proxying, I can connect to apache and then redirect to Zope - but when I do, the lock icon in the browser switches back to "insecure", the Browser complains that there is unencrypted text being passed and it doesnt *look* like I am in SSL anymore. Obviously, I am missing something here -s if anyone can help I would appreciate it... On another semi-related issue - how does one pass the certificate generated by apache to the Zserver for a kind of "single signon"? TIA WPH

3 2

Problem with Topic Index
by Vladimir Petrovic 03 Jun '03

03 Jun '03

I have a trange problem with topic index (on Zope 2.6.1 running on Linux). I created a filtered set with the expression (o.title == 'aa'). I catalogued only one document (this document has title 'aa') and filtered set reports it has one document. Then I went to the Catalog of the ZCatalog, and I removed the catalogued object. All indexes removed the object except this filtered set which still contains one object (ZCatalog itself contains zero objects). And then querying topic index with this filtered set produces KeyError in ZCatalog. It seems like topic index is ignoring unindex operation. Is there anything special that needs to be done with topic indexes ? Vladimir

1 0

Persistence - I know I'm not doing this right...
by Samir Mishra 03 Jun '03

03 Jun '03

Hello All, This is a newbie question, almost embarrassing. I've been storing persistent data in Zope within a Python script. For the sake of simplicity, suppose I want to save values for 2 variables, x & y in Zope, I create a Python script with the following code - return {'x':'value1', 'y':'value2'} When I want to access 'x' & 'y', I call the above script. Q: What's the right way to store persistent objects/data within Zope? TIA Regards, Samir

2 1

caching in zope; tips, tricks, & lessons learned
by Jamie Heilman 03 Jun '03

03 Jun '03

We all care about speed right? Sure we do. If you've been using a RAM Cache Manager to reduce those painfully long render times of wrought by zope page templates or complex zsql methods, there are some things you might not have known which I plan to elucidate here. First, its important to understand exactly what knobs the caching API gives us. Briefly they are, the ability to set a key, value pair in the cache, and the ability to get a value from the cache given a key, and the ability to delete any entries associated with an object. (Its really not unlike a dictionary when you get right down to it.) For the Product author, there are two areas of this API that deserve special focus, control over the "key" used by the cache and control over the peripheral logic dictating the overall "cost" of a cache entry. I'll quickly address the issue of cost first, as its simple - the Cacheable class really doesn't have many concepts of "cost." Objects implementing the Cacheable interface can define a function which influences the mtime (last modification time) calculation of an object, but thats it, there is no other notion of cost logic embedded within the API. This significantly limits what kind of cache replacement algorithms you can implement, but there probably isn't any better alternative given the wide variety of object types in Zope. (If you're confused by my rambling, its OK; let me give a more concrete example. Zope has no parameter in the ZCacheable_set method that lets you tell the associated cache manager, "this data took me 2.453 seconds to generate, so I will assign its time-cost metric a value of 2.453." Thus the cache manager has no way of gauging the relative overhead of flushing a caching entry that took 5 seconds to generate, vs. one that took only 2 seconds. OTOH, I'm not sure if a per-thread job timer is a reasonable thing to ask Cacheable objects to implement or not, or if it would even matter in the long run.) Its worthwhile to note the RAM Cache Manager uses a LFU replacement policy with some tweakable parameters. It could just as easily implemented a LRU policy, one that expires based on entry size, or a hybrid thereof, but there's nothing wrong with LFU; different replacement policies are good in different situations. Control over the 'key' used by the cache manager is arguably just as important as the cost logic, and here the caching API gives a very significant (bordering on limitless) amount of options. Objects are allowed to define a view_name (this is generally a ZClass thing, the view_name being the id of the object which is acting as a method of the ZClass instance), and a keyword dictionary that can contain anything the object author thinks should be used in determining entry uniqueness (dtml methods let you control this through the zmi). Additionally cache managers have access to anything which can be acquired from the object itself. RAM Cache Managers make good use of all this flexibility but there are some gotchas I think people should be aware of which is why I'm writing this whole mess in the first place. The first gotcha, is that RAM Cache Managers let you define a list of variables to give to REQUEST.get(), the values of which will be added to the cache entry key--this is both good and bad. Its good because it lets you, for example, make sure AUTHORIZED_USER is part of the entry key, ensuring user-specific data doesn't leak. Its bad because it presents the user with a tempting but dangerous way to cache the results of query submissions. If you allow RAM Cache Manager to add, say, the result of REQUEST.get("foo") to the key of an entry, its entirely possible you just opened yourself up to a trivial DoS attack as you've just allowed clients to store up to whatever your configured variable size limits are (probably the max POST query size for most people) in the cache. Even if you have POST queries turned off and only allow the normal 8k GET request, thats still quite a bit of data that the client can now re-insert over and over again into your cache. RAM Cache Manager has no size limits. There are a number of ways this can be solved, (message digests, length limits, etc.) but in the interim just be aware the problem exists and use RAM Cache Managers with appropriate caution. The second gotcha is specific to Page Templates. DTML methods let you choose which bits of their namespace should placed in the keywords dictionary used by the caching API to indicate entry uniqueness. By default, DTML methods send an empty dictionary to the ZCacheable_set function. Page Templates OTOH always send a dictionary that contains some problematic entries: 'bound_names':{'traverse_subpath': [], 'options': {'args': ()}, 'user': Anonymous User}, 'here':<Folder instance at 896be80> The bound_names entry hasn't been a problem yet for me as far as I've been able to tell... but as RAM Cache Manager will str() every key and value pair in the keywords dict, there's a possibility the order of bound_names' keys could change between str()s and accidently create a cache miss. That said, I've never seen it happen. What I have seen happen though, is the 'here' value can change if enough memory activity occurs. I was seeing some confusing activity whereby I'd cache a large bundle of page templates, by running wget over a tree, then I'd run wget over the same tree and the cache would miss for every single one. At first I thought it was a cleanup timing issue but after examining it more carefully I saw that in fact the keys were changing because the first time through the value of 'here' would be <Folder instance at 8222222> and the second time it would be <Folder instance at 8333333>. (numbers obviously falsified) This tends to make RAM Cache Managers less effective than the should be at caching page templates. If there's a way to alter the dictionary page templates send to the caching infrastructure, its not obvious what it is. (I haven't looked particularly hard though.) But the lesson here is clear, grabbing the str() of an object isn't always a cache friendly tactic. That said, I've written a cache manager which avoids both of these predicaments. Its called MemoryCache, its on zope.org, but before you rush off to try it out be forewarned it requires python 2.2 and an enhanced version of the OFS/Cache.py module (available in collector issue 911). I solved the first problem by using message digests, and enforcing a cache size limit, and the second problem by allowing for extreme flexibility in determining key uniqueness by giving cache controllers the ability to write their own key aggregation function in restricted python. My tests indicate it uses far less memory when caching things that tend to generate redundant data (like tree widgets) and that the overhead of digesting everything was basically small enough to be lost in the statistical noise of my benchmarks. It does however use a LRU replacement policy, rather than an LFU like RAM Cache Manager, so may not be a suitable replacement for a RAM Cache Manager in all situations. -- Jamie Heilman http://audible.transient.net/~jamie/ "You came all this way, without saying squat, and now you're trying to tell me a '56 Chevy can beat a '47 Buick in a dead quarter mile? I liked you better when you weren't saying squat kid." -Buddy

1 0

Call for Participants for Zope 3 Sprint before Europython 2003 in Belgium
by Grégoire Weber 03 Jun '03

03 Jun '03

Hi everybody, the Zope 3 sprint in Louvain-la-Neuve at the local university before the Europython conference is now definitive. When: Three days from june 22 to 24, 2003 (sunday to thursday). Registration: If you decided to participate the sprint just drop me a e-mail and register your name and e-mail in the sprint wiki (see below). Accomodation: Take care to decide on participation as early as possible. Quick reservation is needed because there seems to be only little place to stay in Louvain-la-Neuve! If possible, plan to arrive on saturday evening/night. This way we don't lose time on sunday morning. Godefroid will help us moving from Louvain-la-Neuve to Charleroi Wednesday morning. So it would make sense to book the night in Louvain-la-Neuve from thursday to wednesday also. Sprint Coaches and Topics: Experienced people like Steve Alexander, the codeworks people (Albertas Agejevas, Marius Gedminas, Viktorija Zaksiene), Martijn Faassen and possibly Godefroid Chapelle will act as Coaches. Topics will be determined definitively during the next weeks. Sprint Preparation: There is a good description about how to prepare for a sprint on <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/CMSprintPr…> Further information: <http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/EuroPython…> More infos to come. Questions? gregoire.weber at switzerland.org regards, Godefroid Chapelle and Gregoire Weber _____________________________________ Grégoire Weber Rigistr. 31 CH-8006 Zürich Switzerland phone: +41-(0)1-361 66 11 mobile: +41-(0)79-44 11 457 mailto:gregoire.weber@switzerland.org

2 1

RE: [Zope] Storing Data inside Zope
by sean.upton＠uniontrib.com 02 Jun '03

02 Jun '03

Perhaps I'm off-base here, but I wonder if Archetypes might be appropriate (schema-driven content types with pluggable storage backends for attributes, custom widgets, validation, FTP/DAV marshalling, etc)... http://sf.net/projects/archetypes http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/archetypes/Archetypes/docs/ Sean > -----Original Message----- > From: Dennis Allison [mailto:allison@sumeru.stanford.EDU] > Sent: Monday, June 02, 2003 2:27 PM > To: Tom P. Allred > Cc: 'J Cameron Cooper'; zope(a)zope.org > Subject: RE: [Zope] Storing Data inside Zope > > > > Sorry if this has already been suggested in this thread.. > > It seems to me that you are really in need of a hybrid system > which uses > both Zope and a relational data base. Zope and some specialized > products to manage the front-end and query interfaces, and a > relational database to handle complex queries. > > MySQL, at least the recent versions, supports full text > indexing as well > as the usual SQL relational queries. I suspect that would be > invaluable > for your application. > > You might consider (if your budget allows) incorporating a > google-style > search engine. > > > > > I'm using the Client/Cases/Detail/etc granularity and > making each record a > > single object (using File objects for testing) in an > appropriate data folder > > (e.g. Case/Data, Client/Data or maybe Data/Client, > Data/Case). I create > > properties with field names and assign them the field > values. That way I > > can use the ZMI to create/clone/edit/delete 'data records' easily in > > addition to my programs. I'm using Formulator so I can tap > the field > > definitions programmatically to get the info needed to > define the File data > > object properties appropriately for each field. I thought > about making a > > template File data object and cloning it as needed but I > haven't figured out > > how to do that yet. Besides the dynamic Formulator > approach lets me change > > form AND 'table' definitions in one spot which is nice for > development. > > > > Being shiny new to Zope I'm still grasping the building > blocks but thought > > ZClasses sounded like what might be useful for this. > Thanks for letting me > > know that might not be a good idea. I'll continue to fumble through > > alternate approaches. It's painfully slow but at least > it's interesting. > > > > Thanks to all who have contributed help, > > Tom P. Allred, MCSE, RHCE, CCNA > > Allred Data Consulting, Inc. > > 704-633-0311 x826 > > www.AllredData.com > > Tom(a)AllredData.com > > > > > _______________________________________________ > Zope maillist - Zope(a)zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) >

1 0