- Zope-Dev - Zope lists

Character codes
by Alexandre Ratti 11 Nov '99

11 Nov '99

Hello, when displaying the raw content of a list with this code: <dtml-call "REQUEST.set('myOptionList',['é', 'à', 'ù', 'è'])"> <dtml-var myOptionList> (The list values are extended characters for French accented letters. They may not display correctly in your mailer.) I get this output: ['\351', '\340', '\371', '\350'] So internally these characters are mapped to character codes. Where can I find reference information about this mapping system? Thanks. Alexandre

2 1

Re: [Zope-dev] Re: [Zope] DTML Syntax contd. + rant
by Simon Michael 11 Nov '99

11 Nov '99

Fully agree with this and the original posting.. after a year on and off zoping, DTML is still hard. And while I may be using it the "wrong way", as it stands every other zope beginner will do the same. What can be done ? NB I post these 2 cents to stimulate ideas and don't imply that DC folk haven't already thought through this whole issue thoroughly. 1. documentation Documenting high-level design patterns and low-level code snippets will definitely help, but I foresee an explosion of recipes. You know something is not optimal when you find yourself saving every useful snippet because you know it will be hard to reinvent. 2. clearer partitioning Clearer partitioning of functionality should clarify the learning curve and reduce frustration. 3. something else What's the ideal situation from user's point of view ? <handwaving> I've heard smalltalk, particularly squeak smalltalk, described as "liquid" and "transparent", meaning you can change things from the application level right down to virtual machine and it's completely consistent. There are no steep jumps in the learning curve requiring a new mindset, language, etc. I'm also reminded of Lucent's inferno where /*everything*/is/in/one/access-controlled/namespace. For zope to be more like this (it already goes a long way), I guess it would be python all the way, with a single conceptually-simple namespace that is always available, from bottom to top. Python is too hard for some ? Keep a limited DTML as an alternative. But to python users we will say: forget dtml; use python, use this single zope namespace, and you can do anything you want, things will work as you expect. Is this (a) what we have already ? (b) attainable ? (c) unrealistic ? Comments welcome. Regards, -Simon

2 1

ngDTML (aka "PSP" or ZScript?) proposal
by Phillip J. Eby 11 Nov '99

11 Nov '99

I have a proposal for a template language which (hopefully) won't be too difficult to create, building as it does on existing works. My proposal is specifically addressed towards making a scripting language that is amenable to learning, doing, and performance, at the expense of being itself valid HTML or XML, and sometimes at the expense of brevity of expression. The syntax I propose we steal from ASP+Python, with one minor alteration. I already have code (written almost 3 years ago and not interfaced to Zope) which parses this syntax, including the necessary machinations to compile out "end if", "end for", etc. and reformat the Python as indented, using the end statements as block delimiters. Such delimiters are necessary when embedding Python in arbitrary text, both for clarity and because indenting is a royal pain in embedded script. Anyway, a basic page might have something like: <% from _ import id, somesqlquery %> Information for <%= _(id) %> <table> <% for record in _(somesqlquery): %> <tr> <td><%= _(record.name1) %></td> <td><%= _(record.name2, fmt="%05.2d") %></td> </tr> <% end for %> </table> Let's go through this and look at the interesting bits. First, we use the name '_' to represent the current Zope namespace. Importing variables from it, or using it as a dictionary always returns those objects *uninterpreted*. However, calling _ with an unnamed first parameter will attempt to execute the object (if it is callable), and then take any keyword parameters as "var"-style formatting arguments. Let's get more specific about the execution part. The execution algorithm is: Push locals() onto the namespace stack, then: 1) If the object has isDocTemp==1, return the result of calling the object with (None,namespace) 2) Otherwise, use ZPublisher's "mapply" function so that the namespace stack can be passed through to the called object's parameters. Thus, if you call a Python function, SQL method, etc., its arguments can automatically be filled in for you as a timesaver. Note that one is under no obligation to use _() to render an object, and that if you want to prevent calling an object you wish to render in its "raw" form, you can simply use str() or repr() or `` on it first. (e.g. _(str(something),fmt="*%s*")) More interesting bits. The Python namespace takes precedence unless you specifically import or otherwise retrieve an object from _. _ itself, REQUEST and RESPONSE, however, are already in the namespace when your method starts, in addition to any parameters defined by the method (ala SQL methods.) There are some special aspects of how import should work. Importing a module should attempt to find an object of that name in the _ namespace, and check the current user's permissions for that object (taking proxy roles into account) before allowing the import. This would allow the use of "PythonModule" objects which you add to a Zope hierarchy and set permissions on. They would import modules only from the "safe" extension and product directories. Of course, any modules generally considered safe could be imported without a lookup/permissons check. Now on to the compilation and execution model. An ngDTML script is rewritten as straight Python code, replacing all <%= expr %> blocks with _.write(expr) and all non-Python text as _.write('text') statements. The resulting code is compiled, and the important aspects of both DTML and Evan Simpson's bytecode transforms and alteration of __builtins__ are performed, to render the resulting code safe for execution. The code block is set up as a function which does a "return _" if execution falls off the end of the script. (Thus, a method can return something instead of writing output, ala the DTML "return" tag.) The _ object would be able to render itself as a string by returning the accumulated writes, and thus when an ngDTML object is called by ZPublisher, its output will be properly sent to the RESPONSE. Whew. It's a long and tedious list of things that have to be implemented. However, it is clearly able to be implemented, and all the pieces already exist, although some bits may have to be re-implemented in order to be integrated. To sum up, the advantages of this language are: 1. It's Python, so the basic syntax is clean and simple. Plenty of books are out there on Python, and when it comes right down to it, we've been continuously extending DTML trying to make it more like Python, so why not just go there? 2. It's based on a well-established embedded scripting paradigm which is used in numerous other application environments including the giants ASP and PHP. It is easier to teach to someone coming over from one of those environments, and many editing tools can already cope with <% %> syntax. 3. All the bits of DTML which are really handy, such as auto-calling, namespace passing, formatting utilities, etc., are all tidily packaged into the "_" swiss-army knife, and there is no confusing Perl-ish automagic anywhere. (Note, by the way, that using _.string, _.None, etc. is not valid in ngDTML; these things should be imported with import. _ is for generally useful ngDTML utilities only.) What about disadvantages? Well, there's no "in" tag and all its goodies, although these could be conceivably implemented using an object that wrapped a sequence for you, and maybe even fitted in as another capability of the '_' object. I haven't given this one a great deal of thought yet. Also, doing things like the tree, sendmail, and sqlgroup tags are tricky unless you can ask the _ object to push and pop its write() buffer; then you can do something like: <% try: _.push() %>From: <%= sender %> To: <%= string.join(recipients,',') %> Subject: test Hi! <% somehost.sendmail(recipients,str(_)) finally: _.pop() %> Which, I have to admit is a lot messier than <dtml-sendmail></dtml-sendmail>. On the other hand, the above can also be written: <% somehost.sendmail(recipients,_(message)) %> Since one can define functions within the body of an ngDTML block, and pass them to utility routines, dtml-tree and other things that want blocks to render could be passed functions. This leaves text-transforming tags like sqlgroup, which are harder to manage in this form. Anyway, other than these points, I believe I have described here a language which is capable of doing anything the DTML core can do, while being much clearer and more consistent, and at the same time practical to implement. Comments, anyone?

1 0

Passing arguments to external methods
by Emmanuel Rousselle 11 Nov '99

11 Nov '99

I'm currently trying to play with external methods using the How-To written by "brian". cf. http://www.zope.org/Documentation/How-To/ExternalMethods When I'm trying to pass the REQUEST object to my external method, Zope complains: Zope has encountered an error while publishing this resource. Error Type: TypeError Error Value: not enough arguments; expected 2, got 1 The method definition looks like "def helloWorld(self, REQUEST)". And the call to the external methods looks like "<dtml-var "helloWorld(REQUEST)">". So it looks to me like Zope is asking for the "self" variable, which is a bit strange since this argument is supposed to be implicit. Version details: Zope version: Zope 2.0.1 (binary release, python 1.5.2, win32-x86) Python version: 1.5.2 (#0, Jul 30 1999, 09:52:18) [MSC 32 bit (Intel)] System Platform: win32 -- Emmanuel

1 0

Zope-beta failure on OSF/Alpha
by Janko Hauser 11 Nov '99

11 Nov '99

I tried for the first time to install Zope on an alpha machine. The compilation went fine, I can start zope and get the initial screen, but if I try to get the management screen zope breaks with a stack overflow and restarts itself. I can ftp into ZServer and get and put files. Anything I can do to investigate this further? Has someone else Zope running on an alpha machine? __Janko

2 1

Re: [Zope] Locks?
by gtk 10 Nov '99

10 Nov '99

[slowly moving this to zope-dev] > I've got a situation where I'd like to be able to have a mutex lock. > Unfortunately, I can't use Lock objects out of the thread module as they > appear to be unpicklable. Code your own module (for my Shrubbery product I called it ShrubLock) which has its own acquire() and release() calls. If you need lots of locks, hack extra stuff into the module and pickle something like an ID. I considered using an MD5 hash of the object URL. Note that this is NOT enough to avoid Z2 CONFLICT errors, as Zope does enough stuff after leaving the methods in your object that there's plenty of time for another transaction to start on the same object. This seems to be a fundamental problem with Zope and needs some serious work on Zope's guts to be resolved. DC apparently advise anyone with write-intensive needs to use non-ZODB storage. Unfortunately, as far as I can tell, "write-intensive" devolves to *any* two modifications at once if you're doing stuff like trying to maintain a ZCatalog of your web site by having your CatalogAware objects call reindex_object(). ZCatalog fans, you better use a cron job to trigger regular re-indexes if your site is going to be at all busy. Regards, Garth. -- <gtk(a)well.com>

1 0

Using DTML is too difficult (was: [Zope] DTML Syntax contd. + rant)
by Rik Hoekstra 10 Nov '99

10 Nov '99

<snip> Martijn, I _really_ like Zope, but I am afraid I have to agree with you. I'll add some more rant. I believe it's important to stress that this is _not_ meant as criticism per se, but only to point out what even for people who have used ZOpe for almost a year are major flaws in its usability. And I can vote for Martijn, that he's not stupid; I myself have the excuse of not coming from a computer science background, but still I rabble on. > Philosophical end rant: > > If I ever get the time I'd be tempted to work on a DTML 'cleanup' > project. DTML is currently *far* too much like Perl and not enough like > Python. *some* easy things are very easy, but as a consequence some > other easy things become far too hard, or at least look far too > complicated. You can spell the same thing in too many ways. The > community encourages and praises additions for convenience (like the > 'default' option, or the new extended &entity; syntax) but as a result > DTML loses its conceptual integrity. It becomes too big and not easy to > understand. > > I understand the argument that DTML shouldn't be used for complicated > purposes, and that you should use Python. This is fine and good, but in > practice people *do* use DTML for complicated purposes. ZClasses in fact > encourage this. The point is that it's not always clear when things are complicated. And using python for complicated tasks is not exactly a piece of cake either, for a number of reasons. - writing products in python is complicated and mostly not worth it - writing python base classes for ZClasses is easier, but using them in the Z framework requires quite a bit of understanding of ZClasses and the Zope framework in general - External methods are much easier, but they have their own disadvantages: (1) it is less than obvious how they interact with the dtml methods/namespace they are called from (2) they are not easily transported to another machine, because you need physical access to the filesystem for that (I understand that that's the only way) > > DTML should be more like Python. Yes > Luckily the Zope framework does allow > new objects to be plugged in using something else than DTML, so not all > hope is lost. :) But it should be easier to use this stuff (do I hear the word developer documentation??? uuummm) > > Feedback on this rant would be appreciated. My 2 cents. Rik

6 7

PostgreSQL and Zope (was Re: [Zope] Summary of DB Discussion)
by Ross J. Reedstrom 10 Nov '99

10 Nov '99

On Tue, Nov 09, 1999 at 06:31:19PM -0600, Karl Fast wrote: <snip> > > - There are DA's for MySQL and PostgreSQL, the two open source databases > (yeah, yeah, MySQL isn't truly open source--although one of the older > version is). They work, although both are still in development (is this > active?) and some people have had troubles getting them to compile. > This seems especially true of the MySQL DA judging from pleas for help > sent to the zope list, although this may be more of a reflection of > MySQL's popularity over PostgreSQL. Since there have been a number of posts recently in the "works for me" catagory, I like to think it reflects fewer problems. ;-) > > - DC is looking for someone in the zope community to take ownership of > the MySQL DA (what's the status of the PostgreSQL DA?) <snip> > Other have addressed some of these points. Since the PostgreSQL DA is on my Zope member page, I thought I should comment on the current state of development. I recently took some vacation: just before I left, the original author of the ZPyGreSQLDA resurfaced in email. Since I've been back, I haven't caught up at paying work, so progress on the DA has suffered. The biggest problem has been that my project at work has gone completely ColdFusion (Zope wasn't there early enough), although we do use PostgreSQL. So I haven't been able to justify much daytime work on it. The biggest problem is probably that I've written many more lines of Tcl than Python, so I'm climbing more than one learing curve at a time. (What I could really use is some good open source time and project management software. Anyone have any recommendations? And what happened to the ZopeCalendar <-> PalmPilot conduit project? ) My experience with the PostgreSQL developers and mailing lists are very similar to the Zope lists: lots of clueful, helpful people (some of them are the same people ;-) Bugs exist (what software is perfect?), but get diagnosed and fixed very rapidly. Features are added (and limitations removed) on a weekly basis. I get the feeling that the pgsql project has turned a corner in the last year or two: the code base has been cleaned up, making it easier to understand, and contribute code. New developers are being attracted, and quickly becoming contributers. It feels like it's reaching critical mass. The pgsql-hackers list reminds me of the linux-kernel list from about '92 or so, in that respect. If you haven't looked at PostgreSQL since it's Postgres95 days, it's definitely time to look again. To get this back on topic for the Zope lists, in particular, I think DC should take a serious look at it: the licenses are compatible (pgsql is Berkeley software, after all) and the corporate attitudes are compatible. (The core developers recently formed PostgreSQL, Inc. and are learning how to be a business, as well as a project: could probably use some pointers from you folks ;-) As another poster mentioned, pgsql is an EORDBMS: Extensible Object Relational DBMS, and not only because you get the source. Building a ZStore for the ZODB on top of PostgreSQLs object features should be easier that relying totally on relational calculus, and might reveal some interesting synergies. If I could unleash the crystal ball (rose tinted), I can see future where Zope and PostgreSQL are the defacto next generation tools for web applications, taking over from perl and MySQL as more critical, core business functions move to the web. Boy, now that I've expressed all that in words, I better get cracking on that DA, shouldn't I? Now that 2.0 seems to have settled down a little, (and 2.1 is in beta) any one want to take a crack at helping? Patches always welcome! Ross -- Ross J. Reedstrom, Ph.D., <reedstrm(a)rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005

2 1

Re: [Zope] Z39.50
by Fredrick Rybarczyk 10 Nov '99

10 Nov '99

Hi Rik, I'm sinking into deeper into a deep pit of dung timewise right now, but I'd really like to do something like this. I also think that zope provides an excellent base for a z39.50 gateway. Most of the machinery is already there. I will try to sketch out a workplan in the beginning of next week (I'm working part-time right now.) but I think these are the main issues of a simple service: 1. Define a way to map a search form query to z39.50 queries. That will require knowledge about the target z39.50 servers so we would need some sort of tool here. 2. Create persistent connections to the target servers.. ..shouldn't be too hard. 3. After sending out the query we need to wait for the result sets and sort them together according to their ranking and user preferences. 4. Present the results. 5. Lots of stuff that I either can't express or type or be bothered with right now :-) 0. Todo all this we need to create a solid session concept -possibly with threads?!? -1. I'm not a crackshot zoper but I'm quite handy with python and c/c++ There aren't much python code out there that I know of but there's a lot of stuff in c:ish languages. Chris, you wrote: """A couple years ago, I worked iwth the freewaissf client libraries, and they're not too bad... I can imagine you might be able to create a new database adapter that would work, similar to how we do LDAP. If you need some ideas, let me know.""" Well, we do need ideas, toss in a few solid concepts and we'll hear you out. Best regards, Fredrick. PS. I'll have a talk about this with the z39.50 dudes around netlab (my work that is).. Rik Hoekstra wrote: > Hello Fredrick, > > Any news on the Z39.50 front? Something from Chris Petrilli > I think Zope could be an ideal vehicle for a Z39.50 gateway, especially when > you want to combine them with other (database) searches, but I have never > quite found a place to start with - doing it all from the bottom up always > seemed a mountain I could not climb - especially as I have not much > experience about Z39.50 and it's so complicated (I do know quite a bit more > about python and Zope, though). > > I have been looking at the research area of your website, and was quite > impressed with the projects there. It seems you would have interesting ideas > on the integration of a z39.50 gateway in Zope. > > greetings, > > Rik Hoekstra > > Faculty of Social Science Library > Leiden University > email: hoekstra(a)fsw.leidenuniv.nl -- +-----------------------------------------------------+ | Fredrick Rybarczyk voice +46 46 222 9369 | | Research Engineer fax +46 46 222 3682 | | LUB NetLab http://www.lub.lu.se/~fredrick/ | +-----------------------------------------------------+

1 0

PROPOSAL: New SQLDA functionality
by Evan Simpson 10 Nov '99

10 Nov '99

Steen Suder wrote: > I'm using PostgreSQL 6.5 and Zope 2.x on RH6. > > When inserting a new record in database I'd like to access the return OID > from the INSERT query from within my ZSQL method. > > That is: if it returns anything... psql does return the OID. > > How do I do? Patch ZPyGreSQLDA :-( Since OIDs (and their return from INSERTs) aren't a generic SQL feature, there isn't any general DA support for them. You would have to hack up something in ZPyGreSQLDA to return the OID as thought it were a SELECT result. I've been thinking that it would be useful to change the standard DA machinery a bit to allow individual adapters to return arbitrary data for each SQL statement. Some kind of 'side-car' attribute containing a sequence of server-specific data. This could be used to allow something like: <dtml-with SQL_put_rec><dtml-var "statement_xdata[0].oid"></dtml-with> ('statment_xdata' would be a list with one element: {'oid': 15678} constructed from the PostgreSQL return value) Cheers, Evan @ 4-am

1 0