[Zope] Re: [ZCommerce] Worthy group agenda: ZOPE dynamic pages and search engines... engines...

[Joerg Herbst]

> CodeIt and other zCommerce folks,
>
>         Thanks for starting this important list.
>
>         The biggest concern we have about moving our sites to Z2/PTK is the
> 'invisibility' factor which can haunt sites based largely on dynamic pages
> rather than static HTML files.
>
>         We have heard anecdotal comments by folks on the general list saying, "No
> sweat, I look at my logs and search engines crawlers and robots index my
> site all the time." Crawling and actually indexing are two different things.
> And which crawlers? There are no blanket easy answers here.
>
>         Search engine indexing can be the lifeblood of certain types of sites. One
> of our sites, http://popculturestore.com, started out as a raunchy
> Fusion-generated collection of template-based pages for the hundreds of
> collectible items in the store's inventory.
>
>         With little effort we started getting orders from all over the world. Each
> hard-coded page was indexed on its own right.
>
>         Then we made The Pop Culture Store MUCH better for human visitors with some
> Perl scripts. Within weeks we dropped off the face of the earth! Search
> engines were not indexing our 'bread and butter' pages.
>
>         Of course the situation should be different with the URL-addressing of ZOPE
> pages. But how different?
>
>         We need an 'ad hoc' study group to create or identify a 'guinea pig'
> e-commerce site which we can use to test and confirm the 'indexability' of
> ZOPE pages with EACH of the MAJOR search engines. (We use First Place
> Software's WebPosition Gold for SE submissions and indexing validation. It
> would be an excellent tool for running such formal tests.)
>
>         Saying that SE indexing of ZOPE sites 'should work' is not going to cut it
> for REAL business folks with REAL sites that depend on REAL visibility.
>
>         We need to find out exactly what our 'search engine friendliness' factor is
> and then have real information with which to respond to this and similar
> questions.
>
>         If we find that major search engines have trouble with ZOPE dynamic pages,
> then we have identified a development requirement to allow ZOPE site
> designers to include a 'shell of static doorway pages' generated and updated
> as part of their template-based site design.
>
>         We believe overcoming customers' objections to dynamic page designs will be
> one of the BIGGEST hurdles in getting ZOPE used for e-commerce.
>
>         We are very interested in collaborating with others to address this issue.
> Due to entrepreneurial commitments, however, we cannot lead such an effort.
>
>         Thanks again for starting this list,
>         --Jim and Timlynn--
>

Hi,

I guess this is not just a ZCommerce Problem, it's a general Problem of dynamically
websites, built for a shop or just for information use. Any project with a lot of
database stored content seems to have this problem. Maybe we can reach some effect with
providing META Information on dynamic pages and with the intensive use of links. But
for a general solution we need to know how robots work when meeting Zope webpages, what
they do and what not.
I'm not that great with such Zope details, but maybe somebody else know, how we can
handle search engines for Zope.

Joerg Herbst