[Grok-dev] Grok deployment story unclear and weird

[Graham Dumpleton]

On May 17, 10:18 pm, Graham Dumpleton <Graham.Dumple... at gmail.com>
wrote:
> On May 17, 8:34 am, Graham Dumpleton <Graham.Dumple... at gmail.com>
> wrote:
>
>
>
> > On May 17, 4:07 am, Laurence Rowe <l... at lrowe.co.uk> wrote:
>
> > > Andreas Jung wrote:
> > > > Hi there,
>
> > > > I am currently totally clueless and confused about the
> > > > deployment story with Grok apps.
>
> > > > As developer I can create my sandbox using grokproject and implement
> > > > my_weird_package based on the generated skeleton. Ok, the application
> > > > is working - now I want to package and distribute my_weird_package.
> > > > What is the suggested approach for doing this?
>
> > > > As a Grok user I want to setup a new Grok server and install
> > > > my_weird_package
> > > > (possibly together with other Grok apps within the Grok instance). As an
> > > > experienced
> > > > Grok user I would use grokproject for creating an empty instance and install
> > > > my_weird_package in some way (in which way?). As a newbie Grok user I would
> > > > expect 'easy_install my_weird_package' to produce a running Grok instance!?
>
> > > > So what is the way doing it the right way - for the Grok developer/package
> > > > and the Grok user?
>
> > > As someone who has never used grok, but would like to be able to install
> > > grok-built applications, I'd feel most comfortable if those applications
> > >   were 'standard' wsgi components, just like the other wsgi applications
> > > I already run (trac, paste, repoze and one day plone). These most likely
> > > run under apachemod_wsgi. There is a tutorial here,http://grok.zope.org/documentation/tutorial/installing-and-setting-up...
>
> > > As far as I can tell, the only drawback to using mod_wsgi compared to
> > > proxying is that there is no way to provide session affinity. Useful
> > > when you want the zodb cache to be relevant.
>
> > There possibly is a way of providing session affinity. First off, use
> > daemon mode with a single process. Obviously all requests come back to
> > the same process so no actual need for it.
>
> > Or, use daemon mode but define multiple daemon process groups of a
> > single process rather than one group with multiple processes. You can
> > then use a couple of different ways to control which of those
> > processes requests go to.
>
> > This has been discussed a little on the mod_wsgi list before, but no
> > one has ever explained a concrete need enough to propose example
> > configuration and/or code of what to do.
>
> > If interested, specify what mechanism you are using to create stick
> > sessions.
>
> I decided to have a play and get together a working example which uses
> cookies.
>
> The Apache configuration for this is below. It is setup with
> assumption that application is mounted at URL of '/sticky', but it can
> be changed as necessary by changing WSGIScriptAlias and the rewrite
> rules. If mounting at root of web site and only application in virtual
> host, you could even remove the RewriteCond's which check the mount
> point. The stickiness will be maintained for a period of 60 minutes
> after last activity, but you can obviously also change that as well if
> need be.
>
> Obviously the downside of sticky sessions is that you loose the
> benefit of request being taken up by whatever process is available and
> are at the mercy of the specific process being used becoming
> overloaded. Rather than just randomly selecting a process, you could
> replace the 'rnd' RewriteMap with a 'prg' RewriteMap which uses some
> executing program to work our which of the process groups is handling
> the least number of active user sessions and route it to that.
>
> # Define multiple process groups each with a single multithreaded
> process.
>
> WSGIDaemonProcess sticky01 processes=1 threads=10 display-name=%
> {GROUP}
> WSGIDaemonProcess sticky02 processes=1 threads=10 display-name=%
> {GROUP}
> WSGIDaemonProcess sticky03 processes=1 threads=10 display-name=%
> {GROUP}
> WSGIDaemonProcess sticky04 processes=1 threads=10 display-name=%
> {GROUP}
> WSGIDaemonProcess sticky05 processes=1 threads=10 display-name=%
> {GROUP}
>
> # Mount our WSGI application at the sub URL of '/sticky'.
>
> WSGIScriptAlias /sticky /usr/local/sites/sticky/site.wsgi
>
> # Lots of rewrite magic to follow.
>
> RewriteEngine On
>
> # Specify a rewrite map which randomly selects one of the process
> groups.
> # The contents of the file should be:
> #
> #    processes sticky01|sticky02|sticky03|sticky04|sticky05
> #
> # That is, an entry for each of the named process groups in
> appropriate
> # format for a random rewrite map.
>
> RewriteMap sticky rnd:/Users/grahamd/Sites/sticky/processes.txt
>
> # Extract the name of the process group from cookie sent in the
> request.
>
> RewriteCond %{REQUEST_URI} ^/sticky(/.*)?$
> RewriteCond %{HTTP_COOKIE} process=([^;]+)
> RewriteRule . - [E=PROCESS:%1,L]
>
> # Validate the name of the process group and if not valid then set it
> # to one of the process groups randomly.
>
> RewriteCond %{REQUEST_URI} ^/sticky(/.*)?$
> RewriteCond %{ENV:PROCESS} !^sticky0[12345]$
> RewriteRule . - [E=PROCESS:${sticky:processes},L]
>
> # Set the cookie so that life time of the cookie always pushed out
> when
> # active. The cookie will expire after defined number of minutes of
> # inactvity and stickiness lost. Using a stickiness timeout of 60
> minutes.
>
> RewriteRule . - [CO=process:%{ENV:PROCESS}:%{HTTP_HOST}:60:/sticky]
>
> # For the WSGI application, indicate that process group should be
> selected
> # based on value for the cookie or where appropriate as set randomly.
> As
> # extra security measure, even though we validate name above, limit
> what
> # process groups could be selected in case there might be others
> configured
> # for same server.
>
> <Location /sticky>
> WSGIRestrictProcess sticky01 sticky02 sticky03 sticky04 sticky05
> WSGIProcessGroup %{ENV:PROCESS}
> </Location>

Here is a slightly better version. This gets rid of checks for mount
point and associates things with Directory that WSGI script file is
in, which is always a better and more secure way of doing things.

# Define multiple process groups each with a single multithreaded
process.

WSGIDaemonProcess sticky01 processes=1 threads=10 display-name=%
{GROUP}
WSGIDaemonProcess sticky02 processes=1 threads=10 display-name=%
{GROUP}
WSGIDaemonProcess sticky03 processes=1 threads=10 display-name=%
{GROUP}
WSGIDaemonProcess sticky04 processes=1 threads=10 display-name=%
{GROUP}
WSGIDaemonProcess sticky05 processes=1 threads=10 display-name=%
{GROUP}

# Mount our WSGI application at the sub URL of '/sticky'.

WSGIScriptAlias /sticky /Users/grahamd/Sites/sticky/site.wsgi

# Specify a rewrite map which randomly selects one of the process
groups.
# The contents of the file should be:
#
#    processes sticky01|sticky02|sticky03|sticky04|sticky05
#
# That is, an entry for each of the named process groups in
appropriate
# format for a random rewrite map.

RewriteMap sticky rnd:/Users/grahamd/Sites/sticky/processes.txt

<Directory /Users/grahamd/Sites/sticky>

# Lots of rewrite magic to follow.

RewriteEngine On

# Extract the name of the process group from cookie sent in the
request.

RewriteCond %{HTTP_COOKIE} process=([^;]+)
RewriteRule . - [E=PROCESS:%1]

# Validate the name of the process group and if not valid then set it
# to one of the process groups randomly.

RewriteCond %{ENV:PROCESS} !^sticky0[12345]$
RewriteRule . - [E=PROCESS:${sticky:processes}]

# Set the cookie so that life time of the cookie always pushed out
when
# active. The cookie will expire after defined number of minutes of
# inactvity and stickiness lost. Using a stickiness timeout of 60
minutes.

RewriteRule . - [CO=process:%{ENV:PROCESS}:%{HTTP_HOST}:1:/sticky]

# For the WSGI application, indicate that process group should be
selected
# based on value for the cookie or where appropriate as set randomly.
As
# extra security measure, even though we validate name above, limit
what
# process groups could be selected in case there might be others
configured
# for same server.

WSGIRestrictProcess sticky01 sticky02 sticky03 sticky04 sticky05
WSGIProcessGroup %{ENV:PROCESS}

</Directory>

Graham