Hi, looks like SVN and viewcvs weren't responding for a couple minutes. I could log into the machine via SSH and Apache serving docs.zope.org still worked. I only noticed: 27059 simon 20 0 98664 6388 4360 R 100 0.2 43327:19 darcs Why do we have darcs running on that machine and more importantly why does it run with 100% cpu load for minutes (or hours?). Hanno
On 4/3/10 4:56 PM, Hanno Schlichting wrote:
27059 simon 20 0 98664 6388 4360 R 100 0.2 43327:19 darcs
Ack, my cunning plan to bog down all but the One True VCS is leaking. Must be more *careful*.. But seriously, ha ha ha, no, I have no idea what that process was doing there, sorry all.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hanno Schlichting wrote:
Hi,
looks like SVN and viewcvs weren't responding for a couple minutes. I could log into the machine via SSH and Apache serving docs.zope.org still worked.
I only noticed:
27059 simon 20 0 98664 6388 4360 R 100 0.2 43327:19 darcs
Why do we have darcs running on that machine and more importantly why does it run with 100% cpu load for minutes (or hours?).
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load? Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAku8n+UACgkQ+gerLs4ltQ6m8ACgjCu/lne/2zQ0YvVbCbfVnipz UkYAn29KBmZe6SzKvmNN++wmCjijqB6e =dk3o -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 4/7/10 17:08 , Tres Seaver wrote:
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load?
I haven't watched it because I don't know when Theuni has set that particular cron job, but I bet the whole repository policy checker is causing a lot of load. There are many other times with high packet loss unrelated to load. Not sure how and why that started. It's a very recent issue. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAku8oVYACgkQRAx5nvEhZLJ3nwCZAW0/DWV2OfJZDj1WRCkhFHCd NboAn1uEXW1OhXjHc99deCpZd+KztafT =YJDS -----END PGP SIGNATURE-----
On 4/7/10 17:14 , Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 4/7/10 17:08 , Tres Seaver wrote:
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load?
I haven't watched it because I don't know when Theuni has set that particular cron job, but I bet the whole repository policy checker is causing a lot of load.
There are many other times with high packet loss unrelated to load. Not sure how and why that started. It's a very recent issue.
Did you check using mtr or a similar tool at which hops the packetloss occur? Currently it looks this this for me: Host Loss% Snt Last Avg Best Wrst StDev 1. gw255.3d12.high5.net 0.0% 123 0.5 1.6 0.5 7.2 1.6 2. vlan159.swcolo1.3d12.xs4all.net 0.0% 123 0.8 0.8 0.4 3.0 0.4 3. 0.ge-1-2-0.xr3.d12.xs4all.net 0.0% 123 0.4 3.8 0.3 114.7 13.7 4. 0.ge-1-2-0.xr1.sara.xs4all.net 0.0% 123 0.9 5.2 0.6 132.8 16.6 5. 10gigabitethernet3-3.core1.ams1.he.net 0.0% 123 1.4 2.5 0.6 11.0 2.9 6. 10gigabitethernet4-1.core1.nyc4.he.net 0.0% 123 86.8 89.2 86.0 97.7 3.5 7. 10gigabitethernet1-2.core1.nyc1.he.net 0.0% 123 95.8 88.6 86.1 99.2 3.3 8. nyiix.nyc.cavtel.net 0.0% 122 87.8 87.8 86.8 94.8 1.0 9. lag-3.cr01.cleyohja.cavtel.com 77.9% 122 15741 15370 15157 16128 278.4 10. ??? 11. xe-1-2-2.er01.asbnvaeq.cavtel.com 41.0% 122 103.6 104.2 102.6 122.8 3.1 12. static-98-140-183-6.hicap.cavtel.net 0.0% 122 103.6 108.6 102.6 298.7 26.3 13. v10.rd1.ric.peak10.net 0.0% 122 103.5 103.2 102.4 114.9 1.1 14. cvs.zope.org 0.0% 122 103.5 103.5 102.4 110.3 1.5 which shows two routers at cavtel that are too busy to return a response, but otherwise a normal connection. Wichert.
On 4/7/10 17:17 , Wichert Akkerman wrote:
On 4/7/10 17:14 , Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 4/7/10 17:08 , Tres Seaver wrote:
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load?
I haven't watched it because I don't know when Theuni has set that particular cron job, but I bet the whole repository policy checker is causing a lot of load.
There are many other times with high packet loss unrelated to load. Not sure how and why that started. It's a very recent issue.
Did you check using mtr or a similar tool at which hops the packetloss occur? Currently it looks this this for me:
thunderbird mangled that badly. There is it properly: http://pastebin.com/raw.php?i=5yWh9KjN Wichert.
On 04/07/2010 05:14 PM, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 4/7/10 17:08 , Tres Seaver wrote:
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load?
I haven't watched it because I don't know when Theuni has set that particular cron job, but I bet the whole repository policy checker is causing a lot of load.
The checker runs at 2:15 CE(ST) for about 90 minutes. However, the script only loads recent revisions and does the working copy checkouts step by step instead of a large run, so I don't think it is aggressive enough to cause troubles. (Can't be sure of course.) Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
On Thu, Apr 8, 2010 at 2:59 AM, Christian Theune <ct@gocept.com> wrote:
On 04/07/2010 05:14 PM, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 4/7/10 17:08 , Tres Seaver wrote:
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load?
I haven't watched it because I don't know when Theuni has set that particular cron job, but I bet the whole repository policy checker is causing a lot of load.
The checker runs at 2:15 CE(ST) for about 90 minutes. However, the script only loads recent revisions and does the working copy checkouts step by step instead of a large run, so I don't think it is aggressive enough to cause troubles. (Can't be sure of course.)
Is it running against the main repo? or a mirror? Jim -- Jim Fulton
On 04/08/2010 03:12 PM, Jim Fulton wrote:
On Thu, Apr 8, 2010 at 2:59 AM, Christian Theune<ct@gocept.com> wrote:
On 04/07/2010 05:14 PM, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 4/7/10 17:08 , Tres Seaver wrote:
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load?
I haven't watched it because I don't know when Theuni has set that particular cron job, but I bet the whole repository policy checker is causing a lot of load.
The checker runs at 2:15 CE(ST) for about 90 minutes. However, the script only loads recent revisions and does the working copy checkouts step by step instead of a large run, so I don't think it is aggressive enough to cause troubles. (Can't be sure of course.)
Is it running against the main repo? or a mirror?
Its running against the main repo - we currently don't have a mirror around, but I'll have one again in the next weeks so if it should be an issue I can relocate that in the future. Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
On Wed, Apr 7, 2010 at 11:08 AM, Tres Seaver <tseaver@palladion.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hanno Schlichting wrote:
Hi,
looks like SVN and viewcvs weren't responding for a couple minutes. I could log into the machine via SSH and Apache serving docs.zope.org still worked.
I only noticed:
27059 simon 20 0 98664 6388 4360 R 100 0.2 43327:19 darcs
Why do we have darcs running on that machine and more importantly why does it run with 100% cpu load for minutes (or hours?).
I'm seeing frequent periods of 50+% packet loss to the machine, and can't even get top to run on it this morning. Any clues about what is causing the load?
We've been having some networking issues over the last week or two that we believe to have been addressed. Jim -- Jim Fulton
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 4/7/10 17:27 , Jim Fulton wrote:
We've been having some networking issues over the last week or two that we believe to have been addressed.
I'm still getting alerts from Nagios 2-3 times a day caused by high packet loss and inability to connect to the services being tested. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAku8pZwACgkQRAx5nvEhZLL6TwCfZ1JDUeVYyriYYbc6xaMRSL1M fvsAn1a4cKBn1IT2+YvgN7bevefhlKyG =wAsX -----END PGP SIGNATURE-----
On Wed, Apr 7, 2010 at 11:32 AM, Jens Vagelpohl <jens@dataflake.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 4/7/10 17:27 , Jim Fulton wrote:
We've been having some networking issues over the last week or two that we believe to have been addressed.
I'm still getting alerts from Nagios 2-3 times a day caused by high packet loss and inability to connect to the services being tested.
Hm, OK. We're continuing to work on this and monitor it. Thanks. Jim -- Jim Fulton
participants (7)
-
Christian Theune -
Hanno Schlichting -
Jens Vagelpohl -
Jim Fulton -
Simon Michael -
Tres Seaver -
Wichert Akkerman