Xymon Mailing List Archive search

TCP Connection Issues?

3 messages in this thread

list Ryan Skadberg · Mon, 30 Jan 2017 17:06:54 -0500 ·
Anyone ever seen this?

I have a few hundred servers in my xymon setup and have about 200 http
checks, as well as some other server side scripts that check things via
tcp.  Recently all of my http and telnet connections will start to flap for
5-10 mins and cause all/most of my http and telnet checks to go red.  I
added a delayred to stop the paging, but I am still getting a wonderful
Christmas tree a few times a day, just yellow now.

It feels like an issue with the number of file handles or maybe the number
of tcp connections, but I can't seem to get the right formula to fix it
up.  Anyone have any suggestions?

Skadz

PS - Running 4.3.17.  Haven't had a time to upgrade in a while (and had
some issues with some of the web changes last time I tried)
list John Thurston · Mon, 30 Jan 2017 13:37:11 -0900 ·
quoted from Ryan Skadberg
On 1/30/2017 1:06 PM, Ryan Skadberg wrote:
It feels like an issue with the number of file handles or maybe the
number of tcp connections, but I can't seem to get the right formula to
fix it up.  Anyone have any suggestions?
It seems a little early to be hypothesizing causes with such a loose description of the failure.

Have you enabled --debug on the xymonnet process? That might tell you if the requests are all going out but the responses are not coming back. Or if there is a significant hicup in name resolution several times a day. Or . . .

If you run 'xymonnet --no-update' when the problem is occurring, do you see anything interesting in the output?

'twould also be helpful to know what operating system and network arrangement you have. Is this all Linux on VMWare with fake nics? Is your server an AX-400 on token ring?


-- 
    Do things because you should, not just because you can.

John Thurston    XXX-XXX-XXXX
user-ce4d79d99bab@xymon.invalid
Enterprise Technology Services
Department of Administration
State of Alaska
list Ryan Skadberg · Wed, 15 Feb 2017 09:01:47 -0500 ·
So, we were able to figure this out on our own.  The fix was to add
"--concurrency=25"
to the xymonnet test and everything got much much better.

Skadz
quoted from Ryan Skadberg


On Mon, Jan 30, 2017 at 5:06 PM, Ryan Skadberg <user-fd74ceebcd5d@xymon.invalid> wrote:
Anyone ever seen this?

I have a few hundred servers in my xymon setup and have about 200 http
checks, as well as some other server side scripts that check things via
tcp.  Recently all of my http and telnet connections will start to flap for
5-10 mins and cause all/most of my http and telnet checks to go red.  I
added a delayred to stop the paging, but I am still getting a wonderful
Christmas tree a few times a day, just yellow now.

It feels like an issue with the number of file handles or maybe the number
of tcp connections, but I can't seem to get the right formula to fix it
up.  Anyone have any suggestions?

Skadz

PS - Running 4.3.17.  Haven't had a time to upgrade in a while (and had
some issues with some of the web changes last time I tried)