AW: AW: AW: [Zope] Source from form result.

4 May 2001


      
...
Did you get enough new  material to write a HowTo? "How To Screen Scrape"
-- Loren
I could. But what do you mean with "Screen Scrape" ???

Marc
...
...
-----Original Message-----
From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Marc
Fischer
Sent: Thursday, May 03, 2001 16:31
To: Tino Wildenhain; Marc Fischer; Danny William Adair; Paula Mangas
Cc: zope@zope.org
Subject: AW: AW: AW: [Zope] Source from form result.
Hey, thanks to all of you, that tried to help me.
Finally I had success with the httplib funktion. ... :-)))
Marc
...
-----Ursprungliche Nachricht-----
Von: zope-admin@zope.org [mailto:zope-admin@zope.org]Im
Auftrag von Tino
Wildenhain
Gesendet: Donnerstag, 3. Mai 2001 22:51
An: Marc Fischer; Danny William Adair; Paula Mangas
Cc: zope@zope.org
Betreff: Re: AW: AW: [Zope] Source from form result.
Hi Marc,
its simply:
<dtml-with "web_client('http://server/',x='a',y='b')">
<dtml-var body>
</dtml-with>
in your case :)
x and y go into this **kw argument. If this is set,
webclient uses POST.
HTH
Tino Wildenhain.
--On Mittwoch, 2. Mai 2001 20:45 +0200 Marc Fischer
<marcbpc@gmx.de> wrote:
...
Hi Tino,
I think, thats it, what I am looking for. Your Example works
fine so far.
But my understanding in python I so low, that I am not able
change it in
my needed way. Perhabs you could tell me how to realize this:
<form action="http://server" method=POST>
<input type=hidden name="x" value="a">
<input type=hidden name="y" value="b">
<input type=submit>
</form>
and show me how to get the source of the result page?! :-)
I would be very happy, ... thanx a lot ...
Marc
...
-----Ursprungliche Nachricht-----
Von: zope-admin@zope.org [mailto:zope-admin@zope.org]Im
Auftrag von Tino
Wildenhain
Gesendet: Mittwoch, 2. Mai 2001 15:37
An: Marc Fischer; Danny William Adair; Paula Mangas
Cc: zope@zope.org
Betreff: Re: AW: [Zope] Source from form result.
Hi Marc,
zope has a "client" deep in its core :)
See lib/python/ZPublisher/Client.py
for more info.
I currently use a external method like this:
...
from ZPublisher import Client
def web_client(url = '', username = None, password = None, **kw):
     '''access http servers'''
     class gen_res:
         __allow_access_to_unprotected_subobjects__=1
     f=gen_res()
     if kw:
         f.headers,f.body=Client.call(url,username,password,kw)
     else:
         f.headers,f.body=Client.call(url,username,password)
     return(f)
And use it like this:
<dtml-with "web_client('http://server/uri')">
<dtml-var body>
</dtml-with>
In Client.py are also methods and functions to map arguments
to a POST request as you wish.
I think, the above could be a starting point for you.
HTH
Tino Wildenhain
--On Mittwoch, 2. Mai 2001 12:12 +0200 Marc Fischer
<marcbpc@gmx.de> wrote:
...
Hi Danny,
first of all thank you for your detailed help!!! That is
nearly, what I am
looking for, but one problem exists:
The page I want to extract (you are right ... its on an external
server) does not work with passing the variables in the url.
I mean ...
http://server?x=a&y=b does not work.
I need to do it with a form like:
<form action="http://server" method=POST>
<input type=hidden name="x" value="a">
<input type=hidden name="y" value="b">
<input type=submit>
</form>
That's my big big problem. Btw, ... its legal :-)))
Marc
> -----Ursprungliche Nachricht-----
> Von: Danny William Adair [mailto:Danny@Adair.net]
> Gesendet: Mittwoch, 2. Mai 2001 01:26
> An: Marc Fischer; Paula Mangas
> Cc: zope@zope.org
> Betreff: Re: [Zope] Source from form result.
>
>
> Hi Marc,
>
> When you submit a form to one of your DTML methods or
documents, all the
> form variables will be available without any further work
to be done
> (REQUEST object, see docs). But if I get you right, you are not
> submitting your form to a method of your own. So things
are a bit
> different here.
>
> As I understand it, you are submitting your form to someone
else's server
> (since otherwise you would produce the output yourself).
Maybe you're
> providing an input form that is directed to a search engine,
or something
> similar.
>
> --------------------------------------------
> <form action="http://server" method=POST>
> <input type=hidden name="x" value="a">
> <input type=hidden name="y" value="b">
>
> <input type="text" name="myInput">
>
> <input type=submit>
> </form>
> --------------------------------------------
>
> If so, then you should take a look at the "KebasData" Product:
> http://www.zope.org/Members/kedai/KebasData
>
> Just instantiate a KebasData object, let's say "results". Take a
> look at the
> product's documentation to see how you can extract the
relevant code from
> the external server's result page. I'll try to get you
started right
> away:
>
> Set the regular expression patterns properly, start with
>
> .*
>
> as the search pattern and leave the rest blank ("url"
will be set
> by a DTML
> method you create, see below), then you'll get the entire
> external page, you
> can change that later to fit your needs.
>
> Then create a DTML method next to it, lets say
"getResults". It could
> look like this:
>
> <dtml-var standard_html_header>
> <h2>RESULTS</h3>
> <dtml-with results>
>   <dtml-call "REQUEST.set('new_url', 'http://server?search=' +
myInput)">
>   <dtml-call "get_matched(new_url)">
>   <dtml-in match>
>     <dtml-var result><br/>
>   </dtml-in>
> </dtml-with>
> <dtml-var standard_html_footer>
>
> This will fetch the page you want to display by calling
> "get_matched(new_url)", you construct that new_url
dynamically, using
> your form variable "myInput". Then it iterates through
the results
> list and spits
> out the parts that matched your query, appending a <br/> after
each match
> (if you started with .* as the pattern, the list will only
have one item,
> holding the complete page).
>
> Oh, right: your DTML method should have a proxy role
that has the
> right to "change KebasData" (or maybe even "View
> management_screens"?) on the KebasData object
"results", since it
> calls its function
> "get_matched", which
> is usually only called by clicking "Initialize!" in the
> management screen of
> the KebasData object.
>
> Now you just send your form to "getResults" (instead of the
> external server
> directly), and let your DTML method do the rest:
>
> --------------------------------------------
> <form action="getResults" method=POST>
> <input type=hidden name="x" value="a">
> <input type=hidden name="y" value="b">
>
> <input type="text" name="myInput">
>
> <input type=submit>
> </form>
> --------------------------------------------
>
> Later, you could put the "getResults" code in a python script,
then it's
> easier to do further processing with your fetched data.
>
> hth,
> Danny
>
> P.S.: Check the legal stuff before extracting other
people's pages ;-)
>
> >>> -----Ursprungliche Nachricht-----
> >>> Von: zope-admin@zope.org [mailto:zope-admin@zope.org]Im
> Auftrag von Marc
> >>> Fischer
> >>> Gesendet: Dienstag, 1. Mai 2001 11.43p
> >>> An: Paula Mangas; Marc Fischer
> >>> Cc: zope@zope.org
> >>> Betreff: AW: [Zope] Source from form result.
> >>>
> >>>
> >>> Hi,
> >>>
> >>> I think there was an understanding problem. I want
to get the
> >>> source code of
> >>> the result that is displayed in the browser, after
klicking on
> >>> the submit
> >>> button. I want to be able to handle this source perhabs
to create
> >>> a dtmlmethod wiht it, or do some string extractions with it.
> >>>
> >>>
> >>>
> >>> > -----Ursprungliche Nachricht-----
> >>> > Von: Paula Mangas [mailto:pamm@students.si.fct.unl.pt]
> >>> > Gesendet: Dienstag, 1. Mai 2001 13:25
> >>> > An: Marc Fischer
> >>> > Cc: zope@zope.org
> >>> > Betreff: Re: [Zope] Source from form result.
> >>> >
> >>> >
> >>> > On Tue, 1 May 2001, Marc Fischer wrote:
> >>> >
> >>> > > Hi,
> >>> > >
> >>> > > I want to get the HTML result from a form into a
dtml method.
> >>> > So I need a
> >>> > > way to handle this result, but HOW.
> >>> > >
> >>> > > The form ist like this:
> >>> > >
> >>> > > <form action="http://server" method=POST>
> >>> > > <input type=hidden name="x" value="a">
> >>> > > <input type=hidden name="y" value="b">
> >>> > > <input type=submit>
> >>> > > </form>
> >>> > >
> >>> > > If I klick on submit, I get the result displayed in the
> >>> Browser, but I
> >>> > > cannot work with it.
> >>> > > Perhaps someone could help me???
> >>> >
> >>> >
> >>> > Hi,
> >>> >
> >>> >
> >>> > I think that, if you try
> >>> > <form action="http://server/the_method_you_want_to_call"
> method=POST>
> >>> >
> >>> > it will work.
> >>> >
> >>> > Paula
> >>> >
> >>>
> >>>
> >>> _______________________________________________
> >>> Zope maillist  -  Zope@zope.org
> >>> http://lists.zope.org/mailman/listinfo/zope
> >>> **   No cross posts or HTML encoding!  **
> >>> (Related lists -
> >>>  http://lists.zope.org/mailman/listinfo/zope-announce
> >>>  http://lists.zope.org/mailman/listinfo/zope-dev )
> >>>
>
_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )