[Checkins] SVN: Sandbox/J1m/resumelb/src/zc/resumelb/zk. Fixed bug running worker in non-test mode.
Jim Fulton
jim at zope.com
Tue Jan 31 12:50:10 UTC 2012
Log message for revision 124267:
Fixed bug running worker in non-test mode.
Added tests to make sure that worker and lb work in non-test mode.
Removed promise of unimplemented feature to allow no access log.
Added ability to pass an object as the access log (mainly for simulation).
Changed:
U Sandbox/J1m/resumelb/src/zc/resumelb/zk.py
U Sandbox/J1m/resumelb/src/zc/resumelb/zk.test
-=-
Modified: Sandbox/J1m/resumelb/src/zc/resumelb/zk.py
===================================================================
--- Sandbox/J1m/resumelb/src/zc/resumelb/zk.py 2012-01-31 10:37:01 UTC (rev 124266)
+++ Sandbox/J1m/resumelb/src/zc/resumelb/zk.py 2012-01-31 12:50:09 UTC (rev 124267)
@@ -38,7 +38,7 @@
worker.zk = zk
if run:
try:
- worker.server.run_forever()
+ worker.server.serve_forever()
finally:
logging.getLogger(__name__+'.worker').info('exiting')
zk.close()
@@ -65,7 +65,7 @@
parser.add_option(
'-l', '--access-log', default='-',
help='Access-log path.\n\n'
- 'Use - (default) for standard output and an empty string to suppress.\n'
+ 'Use - (default) for standard output.\n'
)
parser.add_option(
'-b', '--backlog', type='int',
@@ -141,8 +141,11 @@
spawn= gevent.pool.Pool(options.max_connections)
else:
spawn = 'default'
- accesslog = (sys.stdout if options.access_log == '-'
- else open(options.access_log, 'a'))
+
+ accesslog = options.access_log
+ if isinstance(accesslog, str):
+ accesslog = sys.stdout if accesslog == '-' else open(accesslog, 'a')
+
server = gevent.pywsgi.WSGIServer(
addr, lb.handle_wsgi, backlog = options.backlog,
spawn = spawn, log = accesslog)
Modified: Sandbox/J1m/resumelb/src/zc/resumelb/zk.test
===================================================================
--- Sandbox/J1m/resumelb/src/zc/resumelb/zk.test 2012-01-31 10:37:01 UTC (rev 124266)
+++ Sandbox/J1m/resumelb/src/zc/resumelb/zk.test 2012-01-31 12:50:09 UTC (rev 124267)
@@ -112,8 +112,7 @@
Address to listed on for web requests
-l ACCESS_LOG, --access-log=ACCESS_LOG
Access-log path. Use - (default) for
- standard output and an empty string to
- suppress.
+ standard output.
-b BACKLOG, --backlog=BACKLOG
Server backlog setting.
-m MAX_CONNECTIONS, --max-connections=MAX_CONNECTIONS
@@ -176,12 +175,15 @@
127.0.0.1 - - [2012-01-28 15:29:30] "GET /hi.html HTTP/1.0" 200 226 0.002349
HTTP/1.0 200 OK...
-If we create another worker, it will be seen by the load balancer:
+If we create another worker, it will be seen by the load
+balancer. This time, we're not going to run the worker in test mode.
+We want to make sure that non-test mode works:
- >>> worker2 = zc.resumelb.zk.worker(
+ >>> worker2_greenlet = gevent.spawn(zc.resumelb.zk.worker,
... app, None,
... zookeeper='zookeeper.example.com:2181', path='/test/lb/workers',
- ... address='127.0.0.1:0', run=False)
+ ... address='127.0.0.1:0')
+ >>> gevent.sleep(.01)
>>> len(lb.pool.workers)
2
@@ -240,14 +242,16 @@
>>> print sock.recv(9999) # doctest: +ELLIPSIS
HTTP/1.0 200 OK...
+ >>> sock.close()
+ >>> sock2.close()
+
We didn't get an access log entry in the output, because it's in the
access log.
>>> accesslog.flush()
>>> with open('access.log') as f:
- ... print f.read()
+ ... print f.read(),
127.0.0.1 - - [2012-01-29 14:11:37] "GET /hi.html HTTP/1.0" 200 226 0.001074
- <BLANKLINE>
By looking at the lb's pool's skilled data structure, we can see that
the test request classifier was used.
@@ -255,13 +259,60 @@
>>> list(lb.pool.skilled)
['a', 'b', "yup, it's a test"]
+Let's shut down the workers:
+
+ >>> worker.stop()
+ >>> worker.zk.close()
+ >>> worker2_greenlet.kill()
+ >>> gevent.sleep(.01)
+
+We see that there are fewer workers:
+
+ >>> len(lb.workletts)
+ 0
+
+
+Not that if we looked at the number of pools, it would still be
+2. This is an artifact of the way the test is run. We shutdown/killed
+the servers, but we didn't close the open worker connections. We
+don't have the plumbing to do that and it's only an issue for the tests.
+
OK. now let's shut down the server and lb.
>>> server.stop()
>>> lb.stop()
>>> lb.zk.close()
-And the workers:
+Finally, let's test that:
- >>> worker.stop()
- >>> worker2.stop()
+- The server runs in non-test mode and that
+
+- We can pass an object as the access log parameter.
+ (Used by simulation script.)
+
+ >>> worker_greenlet = gevent.spawn(zc.resumelb.zk.worker,
+ ... app, None,
+ ... zookeeper='zookeeper.example.com:2181', path='/test/lb/workers',
+ ... address='127.0.0.1:0')
+
+ >>> import StringIO
+ >>> accesslog = StringIO.StringIO()
+
+ >>> lb_greenlet = gevent.spawn(zc.resumelb.zk.lbmain, [
+ ... '-a127.0.0.1:0',
+ ... '-l', accesslog, 'zookeeper.example.com:2181', '/test/lb'])
+ >>> gevent.sleep(.1)
+
+ >>> [addr] = map(zc.parse_addr.parse_addr,
+ ... zk.get_children('/test/lb/providers'))
+ >>> sock = gevent.socket.create_connection(addr)
+ >>> sock.sendall('''GET /hi.html HTTP/1.0\r
+ ... Host: h1.com\r
+ ... Content-Length: 0\r
+ ... \r
+ ... ''')
+ >>> print sock.recv(9999) # doctest: +ELLIPSIS
+ HTTP/1.0 200 OK...
+
+ >>> print accesslog.getvalue(),
+ 127.0.0.1 - - [2012-01-31 07:46:31] "GET /hi.html HTTP/1.0" 200 226 0.001165
More information about the checkins
mailing list