Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec. Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of? --- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
I'm curious what about ftp'ing locally 127.0.0.1:ftpport ? still 10k/sec? ----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: <zope@zope.org> Sent: Saturday, February 21, 2004 4:20 PM Subject: [Zope] Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
--- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Hi, I have installed exUserFolder. After creating a user I get the following error when attempting to edit that user: Environment: Zope 2.7.0 egenix-mx-base-2.0.5 psycopg-1.1.11 exUserFolder-0_20_0 Traceback (innermost last): * Module ZPublisher.Publish, line 100, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 40, in call_object * Module Shared.DC.Scripts.Bindings, line 306, in __call__ * Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec * Module App.special_dtml, line 175, in _exec * Module DocumentTemplate.DT_In, line 703, in renderwob * Module DocumentTemplate.DT_In, line 626, in renderwob * Module DocumentTemplate.DT_Util, line 201, in eval __traceback_info__: listUserProperties * Module <string>, line 0, in ? * Module Products.exUserFolder.exUserFolder, line 442, in listUserProperties * Module Products.exUserFolder.pgPropSource.pgPropSource, line 223, in listUserProperties * Module Products.exUserFolder.pgPropSource.pgPropSource, line 206, in loadUserProperties NameError: global name 'time' is not defined An error was encountered while publishing this resource. Error Type: NameError Error Value: global name 'time' is not defined Any ideas of what could be the problem? Thanks, Mike
Michael Long wrote at 2004-2-22 18:15 -0500:
I have installed exUserFolder. After creating a user I get the following error when attempting to edit that user: ... listUserProperties * Module Products.exUserFolder.pgPropSource.pgPropSource, line 206, in loadUserProperties
NameError: global name 'time' is not defined
Almost surely a trivial problem in "pgPropSource": Look at the code in line 206 and import the required "time" object. -- Dieter
Indeed we still only get 10k/sec via ftp to localhost. It is worth noting that FTP upload is full-speed. Also noteworthy is that the CPU on this box is going to 100% when I download via FTP (45% user/55% system). This problem is exhibited, pretty much identically, in Zope 2.6.4 and 2.7. Zope 2.6.2 seems to have critical bugs in its FTP implementation, which means I needed to be off of it yesterday. • Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope. • We are running different versions of Python (2.1 for Zope 2.6.4, 2.3.3 for Zope 2.7) for each of these servers, which may suggests it isn't a Python-related problem, unless there is something we did wrong twice. (And our 2.6.2 is running fine on Python 2.1, although it is a separate install of it than the one 2.6.4 uses). • Everything else runs at a proper transfer rate on this machine - and executing the 2.6.4 and 2.7 installs on the machine where we run 2.6.2 as a process successfully yields the same results - which may suggest that the hardware is fine. Conclusion: I got nothing. Does anyone have any thoughts at all on this subject? Ed On Feb 22, 2004, at 11:51 AM, Bobb wrote:
I'm curious what about ftp'ing locally 127.0.0.1:ftpport ? still 10k/sec?
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: <zope@zope.org> Sent: Saturday, February 21, 2004 4:20 PM Subject: [Zope] Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
--- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
On Wed, Feb 25, 2004 at 01:21:29PM -0700, Edward Pollard wrote:
Indeed we still only get 10k/sec via ftp to localhost.
It is worth noting that FTP upload is full-speed. Also noteworthy is that the CPU on this box is going to 100% when I download via FTP (45% user/55% system).
This problem is exhibited, pretty much identically, in Zope 2.6.4 and 2.7.
Zope 2.6.2 seems to have critical bugs in its FTP implementation, which means I needed to be off of it yesterday.
? Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope.
? We are running different versions of Python (2.1 for Zope 2.6.4, 2.3.3 for Zope 2.7) for each of these servers, which may suggests it isn't a Python-related problem, unless there is something we did wrong twice. (And our 2.6.2 is running fine on Python 2.1, although it is a separate install of it than the one 2.6.4 uses).
? Everything else runs at a proper transfer rate on this machine - and executing the 2.6.4 and 2.7 installs on the machine where we run 2.6.2 as a process successfully yields the same results - which may suggest that the hardware is fine.
Conclusion: I got nothing.
Does anyone have any thoughts at all on this subject?
Dunno. It couldn't hurt to get a couple more data points: - What OS are you running zope on? - What ftp client(s) have you tried? -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's FIST A-THON! (random hero from isometric.spaceninja.com)
On Feb 25, 2004, at 1:33 PM, Paul Winkler wrote:
Conclusion: I got nothing.
Does anyone have any thoughts at all on this subject?
Dunno. It couldn't hurt to get a couple more data points: - What OS are you running zope on? - What ftp client(s) have you tried?
This is all on RedHat 9 servers. Clients are : Dreamweaver 4 (Mac/PC), SmartFTP (PC), WSFTP (PC), command line FTP (Win 2k, XP Pro, Mac OS 10.3, RedHat 9). Ed
Perhaps your server is shaping traffic via iproute2 and tc, and your Zope FTP port is in a class that gets throttled? You might try temporarily putting the Zope FTP on port 21 and seeing if the download rate is still low. -- Fred Yankowski fred@ontosys.com tel: +1.630.879.1312 OntoSys, Inc PGP keyID: 7B449345 fax: +1.630.879.1370 www.ontosys.com 38W242 Deerpath Rd, Batavia, IL 60510-9461, USA
Which OS / processor / memory are you running Edward ? Which network type ? Ethernet/speed, etc. Does the machine have an non-zope ftp server on it? If you use that do you get the same results? I find the 100% cpu spike even stranger than slow performance... (but that's just me ;) ) ----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: "Bobb" <rawbobb@hotmail.com> Cc: <zope@zope.org> Sent: Wednesday, February 25, 2004 3:21 PM Subject: Re: [Zope] Zope 2.7 FTP Bandwidth Limiter? Indeed we still only get 10k/sec via ftp to localhost. It is worth noting that FTP upload is full-speed. Also noteworthy is that the CPU on this box is going to 100% when I download via FTP (45% user/55% system). This problem is exhibited, pretty much identically, in Zope 2.6.4 and 2.7. Zope 2.6.2 seems to have critical bugs in its FTP implementation, which means I needed to be off of it yesterday. • Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope. • We are running different versions of Python (2.1 for Zope 2.6.4, 2.3.3 for Zope 2.7) for each of these servers, which may suggests it isn't a Python-related problem, unless there is something we did wrong twice. (And our 2.6.2 is running fine on Python 2.1, although it is a separate install of it than the one 2.6.4 uses). • Everything else runs at a proper transfer rate on this machine - and executing the 2.6.4 and 2.7 installs on the machine where we run 2.6.2 as a process successfully yields the same results - which may suggest that the hardware is fine. Conclusion: I got nothing. Does anyone have any thoughts at all on this subject? Ed On Feb 22, 2004, at 11:51 AM, Bobb wrote:
I'm curious what about ftp'ing locally 127.0.0.1:ftpport ? still 10k/sec?
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: <zope@zope.org> Sent: Saturday, February 21, 2004 4:20 PM Subject: [Zope] Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
--- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
The 100% CPU spike is likely the cause of the slow performance. I think a reasonable strategy might be to turn on profiling in Zope 2.7 (see zope.conf profile-publisher? key, and Control_Panel -> Debug) to see what falls out of that. I'm sure there's a lot of information about this if you google for "zope profiling". On Thu, 2004-02-26 at 08:26, Bobb wrote:
Which OS / processor / memory are you running Edward ? Which network type ? Ethernet/speed, etc. Does the machine have an non-zope ftp server on it? If you use that do you get the same results? I find the 100% cpu spike even stranger than slow performance... (but that's just me ;) )
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: "Bobb" <rawbobb@hotmail.com> Cc: <zope@zope.org> Sent: Wednesday, February 25, 2004 3:21 PM Subject: Re: [Zope] Zope 2.7 FTP Bandwidth Limiter?
Indeed we still only get 10k/sec via ftp to localhost.
It is worth noting that FTP upload is full-speed. Also noteworthy is that the CPU on this box is going to 100% when I download via FTP (45% user/55% system).
This problem is exhibited, pretty much identically, in Zope 2.6.4 and 2.7.
Zope 2.6.2 seems to have critical bugs in its FTP implementation, which means I needed to be off of it yesterday.
• Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope.
• We are running different versions of Python (2.1 for Zope 2.6.4, 2.3.3 for Zope 2.7) for each of these servers, which may suggests it isn't a Python-related problem, unless there is something we did wrong twice. (And our 2.6.2 is running fine on Python 2.1, although it is a separate install of it than the one 2.6.4 uses).
• Everything else runs at a proper transfer rate on this machine - and executing the 2.6.4 and 2.7 installs on the machine where we run 2.6.2 as a process successfully yields the same results - which may suggest that the hardware is fine.
Conclusion: I got nothing.
Does anyone have any thoughts at all on this subject?
Ed
On Feb 22, 2004, at 11:51 AM, Bobb wrote:
I'm curious what about ftp'ing locally 127.0.0.1:ftpport ? still 10k/sec?
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: <zope@zope.org> Sent: Saturday, February 21, 2004 4:20 PM Subject: [Zope] Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
--- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
The profiler seems to offer absolutely no information of use. I suspect it does not include Medusa performance. The only thing that popped up was a call to FileStorage.py, but that call did not seem to monopolize the CPU. Attached are three logs 1) The FTP session 2) The profiler data for that time period. 3) the TOP data for that period Due to the bugs in 2.6.2, I've had to disable FTP access to the server, which has my clients infuriated. Any help at all with this would be most grateful. The only other option is to got back to 2.6.0, but that means we can't compress the ZODB without truncating it to 4 bytes. Ed 230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. ftp> put big_file.exe local: big_file.exe remote: big_file.exe 227 Entering Passive Mode (127,0,0,1,202,169) 150 Opening Binary connection for big_file.exe 226 Transfer complete. 18817367 bytes sent in 0.288 secs (6.4e+04 Kbytes/sec) ftp> get large_bin.exe local: large_bin.exe remote: large_bin.exe 227 Entering Passive Mode (127,0,0,1,202,156) 150 Opening Binary mode data connection for file 'large_bin.exe' ######################################################################## ######################################################################## ######################################################################## ######################################################################## ######################################################################## ######################################################################## receive aborted waiting for remote to finish abort 426 Connection closed; transfer aborted 226 ABOR command successful. 442368 bytes received in 35.2 secs (12 Kbytes/sec) ftp> exit 221 Goodbye. Here is the Profiler data that covers the timeframe of this activity (where tottime > 0) Ordered by: internal time List reduced from 446 to 100 due to restriction <100> ncalls tottime percall cumtime percall filename:lineno(function) 3 0.280 0.093 0.280 0.093 ApplicationManager.py:183(refcount) 21 0.040 0.002 0.050 0.002 Connection.py:598(_set_ghost_state) 812 0.040 0.000 0.040 0.000 DT_HTML.py:23(search) 21 0.040 0.002 0.040 0.002 FileStorage.py:651(_load) 1007 0.020 0.000 0.020 0.000 HTTPRequest.py:1211(__getitem__) 16 0.020 0.001 0.310 0.019 DT_In.py:618(renderwob) 114 0.020 0.000 0.040 0.000 HTTPRequest.py:1231(keys) 473/197 0.020 0.000 0.020 0.000 DT_Util.py:341(parse_params) 22 0.020 0.001 0.030 0.001 PersistentExtra.py:23(bobobase_modification_time) 96/39 0.020 0.000 0.050 0.001 DT_String.py:244(parse_close) 117 0.020 0.000 0.040 0.000 DT_Var.py:170(__init__) 1638 0.020 0.000 0.020 0.000 DT_Var.py:183(<lambda>) 725 0.020 0.000 0.030 0.000 DT_String.py:95(_parseTag) 181 0.010 0.000 0.010 0.000 DT_Util.py:229(name_param) 24 0.010 0.000 0.010 0.000 DateTime.py:352(_julianday) 2 0.010 0.005 0.010 0.005 ApplicationManager.py:285(version_txt) 134 0.010 0.000 0.010 0.000 DT_String.py:186(skip_eol) 19/17 0.010 0.001 0.010 0.001 Traversable.py:95(getPhysicalPath) 249 0.010 0.000 0.010 0.000 Connection.py:180(_persistent_load) 77/75 0.010 0.000 0.120 0.002 DT_Util.py:175(eval) 725 0.010 0.000 0.010 0.000 DT_HTML.py:129(start) 1158 0.010 0.000 0.010 0.000 HTTPRequest.py:1090(get) 7 0.010 0.001 0.010 0.001 HTTPRequest.py:148(setServerURL) 8 0.010 0.001 0.010 0.001 posixpath.py:171(exists) 6 0.010 0.002 0.010 0.002 sre_compile.py:229(_mk_bitmap) 725 0.010 0.000 0.010 0.000 DT_HTML.py:152(parseTag) 11 0.010 0.001 0.020 0.002 VirtualHostMonster.py:130(__call__) 8 0.010 0.001 0.020 0.003 DT_String.py:514(read_raw) 59/20 0.010 0.000 0.180 0.009 DT_String.py:195(parse_block) 2 0.010 0.005 0.200 0.100 ApplicationManager.py:204(refdict) 78 0.010 0.000 0.010 0.000 User.py:171(allowed) 11 0.010 0.001 0.010 0.001 HTTPResponse.py:266(setBody) TOP data, where %CPU > 0: PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 15732 apache 25 0 80292 78M 3624 R 99.8 7.7 1:25 0 python2.3 15807 root 15 0 1252 1252 1016 S 0.2 0.1 0:00 0 ftp 1 root 15 0 476 448 424 S 0.0 0.0 0:04 0 init On Feb 26, 2004, at 10:40 AM, Chris McDonough wrote:
The 100% CPU spike is likely the cause of the slow performance. I think a reasonable strategy might be to turn on profiling in Zope 2.7 (see zope.conf profile-publisher? key, and Control_Panel -> Debug) to see what falls out of that. I'm sure there's a lot of information about this if you google for "zope profiling".
On Thu, 2004-02-26 at 08:26, Bobb wrote:
Which OS / processor / memory are you running Edward ? Which network type ? Ethernet/speed, etc. Does the machine have an non-zope ftp server on it? If you use that do you get the same results? I find the 100% cpu spike even stranger than slow performance... (but that's just me ;) )
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: "Bobb" <rawbobb@hotmail.com> Cc: <zope@zope.org> Sent: Wednesday, February 25, 2004 3:21 PM Subject: Re: [Zope] Zope 2.7 FTP Bandwidth Limiter?
Indeed we still only get 10k/sec via ftp to localhost.
It is worth noting that FTP upload is full-speed. Also noteworthy is that the CPU on this box is going to 100% when I download via FTP (45% user/55% system).
This problem is exhibited, pretty much identically, in Zope 2.6.4 and 2.7.
Zope 2.6.2 seems to have critical bugs in its FTP implementation, which means I needed to be off of it yesterday.
• Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope.
• We are running different versions of Python (2.1 for Zope 2.6.4, 2.3.3 for Zope 2.7) for each of these servers, which may suggests it isn't a Python-related problem, unless there is something we did wrong twice. (And our 2.6.2 is running fine on Python 2.1, although it is a separate install of it than the one 2.6.4 uses).
• Everything else runs at a proper transfer rate on this machine - and executing the 2.6.4 and 2.7 installs on the machine where we run 2.6.2 as a process successfully yields the same results - which may suggest that the hardware is fine.
Conclusion: I got nothing.
Does anyone have any thoughts at all on this subject?
Ed
On Feb 22, 2004, at 11:51 AM, Bobb wrote:
I'm curious what about ftp'ing locally 127.0.0.1:ftpport ? still 10k/sec?
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: <zope@zope.org> Sent: Saturday, February 21, 2004 4:20 PM Subject: [Zope] Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
--- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
As you said, I'm afraid that doesn't give much information. The profiler does capture Medusa calls (it captures all Python calls, it doesn't discriminate), but it's not even clear that the issue is really in Medusa. It might be and likely is way up in Zope somewhere. Note that I don't know what it means to sort by internal time, so I'm really not sure how to interpret the profile data. It might be better to sort it descending by total time or cumulative time. Also, it might be more informative to have more than 100 lines of profiling data (500 lines or so would be better). Basically, you're just looking for stuff that takes an inordinate amount of time. The next step if profiling doesn't help is to attach to the running process using gdb while downloading to a sense of where it's looping. I don't remember how to do this off the top of my head but there have been a few maillist threads which discuss it. It also might be possible to run Zope under strace to what's happening when the download is going on. - C On Fri, 2004-02-27 at 11:59, Edward Pollard wrote:
The profiler seems to offer absolutely no information of use. I suspect it does not include Medusa performance.
The only thing that popped up was a call to FileStorage.py, but that call did not seem to monopolize the CPU.
Attached are three logs 1) The FTP session 2) The profiler data for that time period. 3) the TOP data for that period
Due to the bugs in 2.6.2, I've had to disable FTP access to the server, which has my clients infuriated. Any help at all with this would be most grateful. The only other option is to got back to 2.6.0, but that means we can't compress the ZODB without truncating it to 4 bytes.
Ed
230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. ftp> put big_file.exe local: big_file.exe remote: big_file.exe 227 Entering Passive Mode (127,0,0,1,202,169) 150 Opening Binary connection for big_file.exe 226 Transfer complete. 18817367 bytes sent in 0.288 secs (6.4e+04 Kbytes/sec) ftp> get large_bin.exe local: large_bin.exe remote: large_bin.exe 227 Entering Passive Mode (127,0,0,1,202,156) 150 Opening Binary mode data connection for file 'large_bin.exe' ######################################################################## ######################################################################## ######################################################################## ######################################################################## ######################################################################## ######################################################################## receive aborted waiting for remote to finish abort 426 Connection closed; transfer aborted 226 ABOR command successful. 442368 bytes received in 35.2 secs (12 Kbytes/sec) ftp> exit 221 Goodbye.
Here is the Profiler data that covers the timeframe of this activity (where tottime > 0)
Ordered by: internal time List reduced from 446 to 100 due to restriction <100>
ncalls tottime percall cumtime percall filename:lineno(function) 3 0.280 0.093 0.280 0.093 ApplicationManager.py:183(refcount) 21 0.040 0.002 0.050 0.002 Connection.py:598(_set_ghost_state) 812 0.040 0.000 0.040 0.000 DT_HTML.py:23(search) 21 0.040 0.002 0.040 0.002 FileStorage.py:651(_load) 1007 0.020 0.000 0.020 0.000 HTTPRequest.py:1211(__getitem__) 16 0.020 0.001 0.310 0.019 DT_In.py:618(renderwob) 114 0.020 0.000 0.040 0.000 HTTPRequest.py:1231(keys) 473/197 0.020 0.000 0.020 0.000 DT_Util.py:341(parse_params) 22 0.020 0.001 0.030 0.001 PersistentExtra.py:23(bobobase_modification_time) 96/39 0.020 0.000 0.050 0.001 DT_String.py:244(parse_close) 117 0.020 0.000 0.040 0.000 DT_Var.py:170(__init__) 1638 0.020 0.000 0.020 0.000 DT_Var.py:183(<lambda>) 725 0.020 0.000 0.030 0.000 DT_String.py:95(_parseTag) 181 0.010 0.000 0.010 0.000 DT_Util.py:229(name_param) 24 0.010 0.000 0.010 0.000 DateTime.py:352(_julianday) 2 0.010 0.005 0.010 0.005 ApplicationManager.py:285(version_txt) 134 0.010 0.000 0.010 0.000 DT_String.py:186(skip_eol) 19/17 0.010 0.001 0.010 0.001 Traversable.py:95(getPhysicalPath) 249 0.010 0.000 0.010 0.000 Connection.py:180(_persistent_load) 77/75 0.010 0.000 0.120 0.002 DT_Util.py:175(eval) 725 0.010 0.000 0.010 0.000 DT_HTML.py:129(start) 1158 0.010 0.000 0.010 0.000 HTTPRequest.py:1090(get) 7 0.010 0.001 0.010 0.001 HTTPRequest.py:148(setServerURL) 8 0.010 0.001 0.010 0.001 posixpath.py:171(exists) 6 0.010 0.002 0.010 0.002 sre_compile.py:229(_mk_bitmap) 725 0.010 0.000 0.010 0.000 DT_HTML.py:152(parseTag) 11 0.010 0.001 0.020 0.002 VirtualHostMonster.py:130(__call__) 8 0.010 0.001 0.020 0.003 DT_String.py:514(read_raw) 59/20 0.010 0.000 0.180 0.009 DT_String.py:195(parse_block) 2 0.010 0.005 0.200 0.100 ApplicationManager.py:204(refdict) 78 0.010 0.000 0.010 0.000 User.py:171(allowed) 11 0.010 0.001 0.010 0.001 HTTPResponse.py:266(setBody)
TOP data, where %CPU > 0:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 15732 apache 25 0 80292 78M 3624 R 99.8 7.7 1:25 0 python2.3 15807 root 15 0 1252 1252 1016 S 0.2 0.1 0:00 0 ftp 1 root 15 0 476 448 424 S 0.0 0.0 0:04 0 init
On Feb 26, 2004, at 10:40 AM, Chris McDonough wrote:
The 100% CPU spike is likely the cause of the slow performance. I think a reasonable strategy might be to turn on profiling in Zope 2.7 (see zope.conf profile-publisher? key, and Control_Panel -> Debug) to see what falls out of that. I'm sure there's a lot of information about this if you google for "zope profiling".
On Thu, 2004-02-26 at 08:26, Bobb wrote:
Which OS / processor / memory are you running Edward ? Which network type ? Ethernet/speed, etc. Does the machine have an non-zope ftp server on it? If you use that do you get the same results? I find the 100% cpu spike even stranger than slow performance... (but that's just me ;) )
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: "Bobb" <rawbobb@hotmail.com> Cc: <zope@zope.org> Sent: Wednesday, February 25, 2004 3:21 PM Subject: Re: [Zope] Zope 2.7 FTP Bandwidth Limiter?
Indeed we still only get 10k/sec via ftp to localhost.
It is worth noting that FTP upload is full-speed. Also noteworthy is that the CPU on this box is going to 100% when I download via FTP (45% user/55% system).
This problem is exhibited, pretty much identically, in Zope 2.6.4 and 2.7.
Zope 2.6.2 seems to have critical bugs in its FTP implementation, which means I needed to be off of it yesterday.
• Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope.
• We are running different versions of Python (2.1 for Zope 2.6.4, 2.3.3 for Zope 2.7) for each of these servers, which may suggests it isn't a Python-related problem, unless there is something we did wrong twice. (And our 2.6.2 is running fine on Python 2.1, although it is a separate install of it than the one 2.6.4 uses).
• Everything else runs at a proper transfer rate on this machine - and executing the 2.6.4 and 2.7 installs on the machine where we run 2.6.2 as a process successfully yields the same results - which may suggest that the hardware is fine.
Conclusion: I got nothing.
Does anyone have any thoughts at all on this subject?
Ed
On Feb 22, 2004, at 11:51 AM, Bobb wrote:
I'm curious what about ftp'ing locally 127.0.0.1:ftpport ? still 10k/sec?
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: <zope@zope.org> Sent: Saturday, February 21, 2004 4:20 PM Subject: [Zope] Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
--- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
FWIW, I think I tracked this down to a too-small buffer size in the FTP server code. The latest code from CVS (Zope-2_7-branch) allows for much faster download speeds (~ 100X), at least for me. On Fri, 2004-02-27 at 11:59, Edward Pollard wrote:
The profiler seems to offer absolutely no information of use. I suspect it does not include Medusa performance.
The only thing that popped up was a call to FileStorage.py, but that call did not seem to monopolize the CPU.
Attached are three logs 1) The FTP session 2) The profiler data for that time period. 3) the TOP data for that period
Due to the bugs in 2.6.2, I've had to disable FTP access to the server, which has my clients infuriated. Any help at all with this would be most grateful. The only other option is to got back to 2.6.0, but that means we can't compress the ZODB without truncating it to 4 bytes.
Ed
230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. ftp> put big_file.exe local: big_file.exe remote: big_file.exe 227 Entering Passive Mode (127,0,0,1,202,169) 150 Opening Binary connection for big_file.exe 226 Transfer complete. 18817367 bytes sent in 0.288 secs (6.4e+04 Kbytes/sec) ftp> get large_bin.exe local: large_bin.exe remote: large_bin.exe 227 Entering Passive Mode (127,0,0,1,202,156) 150 Opening Binary mode data connection for file 'large_bin.exe' ######################################################################## ######################################################################## ######################################################################## ######################################################################## ######################################################################## ######################################################################## receive aborted waiting for remote to finish abort 426 Connection closed; transfer aborted 226 ABOR command successful. 442368 bytes received in 35.2 secs (12 Kbytes/sec) ftp> exit 221 Goodbye.
Here is the Profiler data that covers the timeframe of this activity (where tottime > 0)
Ordered by: internal time List reduced from 446 to 100 due to restriction <100>
ncalls tottime percall cumtime percall filename:lineno(function) 3 0.280 0.093 0.280 0.093 ApplicationManager.py:183(refcount) 21 0.040 0.002 0.050 0.002 Connection.py:598(_set_ghost_state) 812 0.040 0.000 0.040 0.000 DT_HTML.py:23(search) 21 0.040 0.002 0.040 0.002 FileStorage.py:651(_load) 1007 0.020 0.000 0.020 0.000 HTTPRequest.py:1211(__getitem__) 16 0.020 0.001 0.310 0.019 DT_In.py:618(renderwob) 114 0.020 0.000 0.040 0.000 HTTPRequest.py:1231(keys) 473/197 0.020 0.000 0.020 0.000 DT_Util.py:341(parse_params) 22 0.020 0.001 0.030 0.001 PersistentExtra.py:23(bobobase_modification_time) 96/39 0.020 0.000 0.050 0.001 DT_String.py:244(parse_close) 117 0.020 0.000 0.040 0.000 DT_Var.py:170(__init__) 1638 0.020 0.000 0.020 0.000 DT_Var.py:183(<lambda>) 725 0.020 0.000 0.030 0.000 DT_String.py:95(_parseTag) 181 0.010 0.000 0.010 0.000 DT_Util.py:229(name_param) 24 0.010 0.000 0.010 0.000 DateTime.py:352(_julianday) 2 0.010 0.005 0.010 0.005 ApplicationManager.py:285(version_txt) 134 0.010 0.000 0.010 0.000 DT_String.py:186(skip_eol) 19/17 0.010 0.001 0.010 0.001 Traversable.py:95(getPhysicalPath) 249 0.010 0.000 0.010 0.000 Connection.py:180(_persistent_load) 77/75 0.010 0.000 0.120 0.002 DT_Util.py:175(eval) 725 0.010 0.000 0.010 0.000 DT_HTML.py:129(start) 1158 0.010 0.000 0.010 0.000 HTTPRequest.py:1090(get) 7 0.010 0.001 0.010 0.001 HTTPRequest.py:148(setServerURL) 8 0.010 0.001 0.010 0.001 posixpath.py:171(exists) 6 0.010 0.002 0.010 0.002 sre_compile.py:229(_mk_bitmap) 725 0.010 0.000 0.010 0.000 DT_HTML.py:152(parseTag) 11 0.010 0.001 0.020 0.002 VirtualHostMonster.py:130(__call__) 8 0.010 0.001 0.020 0.003 DT_String.py:514(read_raw) 59/20 0.010 0.000 0.180 0.009 DT_String.py:195(parse_block) 2 0.010 0.005 0.200 0.100 ApplicationManager.py:204(refdict) 78 0.010 0.000 0.010 0.000 User.py:171(allowed) 11 0.010 0.001 0.010 0.001 HTTPResponse.py:266(setBody)
TOP data, where %CPU > 0:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 15732 apache 25 0 80292 78M 3624 R 99.8 7.7 1:25 0 python2.3 15807 root 15 0 1252 1252 1016 S 0.2 0.1 0:00 0 ftp 1 root 15 0 476 448 424 S 0.0 0.0 0:04 0 init
On Feb 26, 2004, at 10:40 AM, Chris McDonough wrote:
The 100% CPU spike is likely the cause of the slow performance. I think a reasonable strategy might be to turn on profiling in Zope 2.7 (see zope.conf profile-publisher? key, and Control_Panel -> Debug) to see what falls out of that. I'm sure there's a lot of information about this if you google for "zope profiling".
On Thu, 2004-02-26 at 08:26, Bobb wrote:
Which OS / processor / memory are you running Edward ? Which network type ? Ethernet/speed, etc. Does the machine have an non-zope ftp server on it? If you use that do you get the same results? I find the 100% cpu spike even stranger than slow performance... (but that's just me ;) )
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: "Bobb" <rawbobb@hotmail.com> Cc: <zope@zope.org> Sent: Wednesday, February 25, 2004 3:21 PM Subject: Re: [Zope] Zope 2.7 FTP Bandwidth Limiter?
Indeed we still only get 10k/sec via ftp to localhost.
It is worth noting that FTP upload is full-speed. Also noteworthy is that the CPU on this box is going to 100% when I download via FTP (45% user/55% system).
This problem is exhibited, pretty much identically, in Zope 2.6.4 and 2.7.
Zope 2.6.2 seems to have critical bugs in its FTP implementation, which means I needed to be off of it yesterday.
• Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope.
• We are running different versions of Python (2.1 for Zope 2.6.4, 2.3.3 for Zope 2.7) for each of these servers, which may suggests it isn't a Python-related problem, unless there is something we did wrong twice. (And our 2.6.2 is running fine on Python 2.1, although it is a separate install of it than the one 2.6.4 uses).
• Everything else runs at a proper transfer rate on this machine - and executing the 2.6.4 and 2.7 installs on the machine where we run 2.6.2 as a process successfully yields the same results - which may suggest that the hardware is fine.
Conclusion: I got nothing.
Does anyone have any thoughts at all on this subject?
Ed
On Feb 22, 2004, at 11:51 AM, Bobb wrote:
I'm curious what about ftp'ing locally 127.0.0.1:ftpport ? still 10k/sec?
----- Original Message ----- From: "Edward Pollard" <pollej@uleth.ca> To: <zope@zope.org> Sent: Saturday, February 21, 2004 4:20 PM Subject: [Zope] Zope 2.7 FTP Bandwidth Limiter?
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
--- Edward J. Pollard, B.Sc Webmaster, University of Lethbridge
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
The 100% CPU spike is likely the cause of the slow performance. I think a reasonable strategy might be to turn on profiling in Zope 2.7 (see zope.conf profile-publisher? key, and Control_Panel -> Debug) to see what falls out of that. I'm sure there's a lot of information about this if you google for "zope profiling".
When the time is spent in ZServer (rather than Zope itself), the profiler will not be able to report this. When I (once) analysed the timing behaviour of an 80 MB file upload, Zope itself used only a small proportion of the complete time. It turned out that "ZPublisher.Client" was responsible for most of the total time (which was used as upload client). It is not designed for large object transfer and forced the computer into swapping. -- Dieter
Edward Pollard wrote at 2004-2-25 13:21 -0700:
... � Dieter does not have this problem in 2.7, which may suggest that the problem is not Zope.
The original report was for an upload. I now repeated it for a download -- this is much faster (probably because the file was already in the ZODB cache). Upload ====== ftp> put pop.log pop.gif local: pop.log remote: pop.gif 227 Entering Passive Mode (127,0,0,1,129,213) 150 Opening Binary connection for pop.gif 100% |*************************************| 2912 KB 40.74 MB/s 00:00 ETA 226 Transfer complete. 2982002 bytes sent in 00:01 (2.69 MB/s) Download ======== ftp> get pop.gif local: pop.gif remote: pop.gif 227 Entering Passive Mode (127,0,0,1,129,215) 150 Opening Binary mode data connection for file 'pop.gif' 100% |*************************************| 2912 KB 29.69 MB/s 00:00 ETA 226 Transfer complete 2982002 bytes received in 00:00 (29.47 MB/s) -- Dieter
Edward Pollard wrote at 2004-2-21 14:20 -0700:
We're experimenting with Zope 2.7, and are currently unable to get FTP transfers to go any faster that 10k/sec. This seems to be some sort of arbitrary limit, as we have plenty of bandwidth and horsepower available. Indeed, we can pretty much make as many FTP sessions as we want simultaneously, they just all go at 10k/sec.
Was there some sort of limiter or throttle introduced somewhere along the lines that I am unaware of?
Unlikely. That's what I see (Zope 2.7b3): ftp> put pop.log local: pop.log remote: pop.log ftp: setsockopt (ignored): Permission denied ---> PASV 227 Entering Passive Mode (127,0,0,1,128,136) ---> STOR pop.log 150 Opening Binary connection for pop.log 100% |*************************************| 2765 KB 33.81 MB/s 00:00 ETA 226 Transfer complete. 2831529 bytes sent in 00:20 (137.11 KB/s) -- Dieter
participants (7)
-
Bobb -
Chris McDonough -
Dieter Maurer -
Edward Pollard -
Fred Yankowski -
Michael Long -
Paul Winkler