Xymon Mailing List Archive search

Message to long

7 messages in this thread

list Vernon Everett · Mon, 7 Aug 2006 09:42:09 +0800 · 
Hi all

Despite my best efforts, my procs page remains purple.
I found this in the hobbitd page - which also remains yellow.
(Server name changed to pluto)

--- snip ---
Latest errormessages:
Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit
524288)
First line: pluto|sunos|sunos
Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447,
limit=262144)
Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated
(n=508456, limit=262144)
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508452,
limit=262144)
Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit
524288)
First line: pluto|sunos|sunos
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448,
limit=262144)
--- snip ---

I am assuming that the output from the ports and procs test is too
large.
How do we make Hobbit happy with the larger message?
This machine is a proxy server, so we can expect many procs and a long
ports output.

Regards
  Vernon
list Lars Ebeling · Mon, 7 Aug 2006 05:52:35 +0200 · 
Have a look at hobbitserver.cfg manpage. MAXMSG_CLIENT

Regards
Lars
▸ quoted from Vernon Everett

----- Original Message ----- 
From: "Everett, Vernon" <user-36f80bd657a9@xymon.invalid>
To: <user-ae9b8668bcde@xymon.invalid>
Sent: Monday, August 07, 2006 3:42 AM
Subject: [hobbit] Message to long


Hi all

Despite my best efforts, my procs page remains purple.
I found this in the hobbitd page - which also remains yellow.
(Server name changed to pluto)

--- snip ---
Latest errormessages:
Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit
524288)
First line: pluto|sunos|sunos
Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447,
limit=262144)
Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated
(n=508456, limit=262144)
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508452,
limit=262144)
Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit
524288)
First line: pluto|sunos|sunos
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448,
limit=262144)
--- snip ---

I am assuming that the output from the ports and procs test is too
large.
How do we make Hobbit happy with the larger message?
This machine is a proxy server, so we can expect many procs and a long
ports output.

Regards
  Vernon
list Vernon Everett · Mon, 7 Aug 2006 12:17:07 +0800 · 
Yep, that was it.
Thanks Lars

Cheers
   Vernon
▸ quoted from Lars Ebeling

-----Original Message-----
From: lars ebeling [mailto:user-1fecd3eafd52@xymon.invalid] 
Sent: Monday, 7 August 2006 11:53 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Message to long

Have a look at hobbitserver.cfg manpage. MAXMSG_CLIENT

Regards
Lars

----- Original Message -----
From: "Everett, Vernon" <user-36f80bd657a9@xymon.invalid>
To: <user-ae9b8668bcde@xymon.invalid>
Sent: Monday, August 07, 2006 3:42 AM
Subject: [hobbit] Message to long


Hi all

Despite my best efforts, my procs page remains purple.
I found this in the hobbitd page - which also remains yellow.
(Server name changed to pluto)

--- snip ---
Latest errormessages:
Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit
524288)
First line: pluto|sunos|sunos
Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447,
limit=262144)
Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated
(n=508456, limit=262144)
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508452,
limit=262144)
Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit
524288)
First line: pluto|sunos|sunos
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448,
limit=262144)
--- snip ---

I am assuming that the output from the ports and procs test is too
large.
How do we make Hobbit happy with the larger message?
This machine is a proxy server, so we can expect many procs and a long
ports output.

Regards
  Vernon
list Vernon Everett · Mon, 7 Aug 2006 16:40:57 +0800 · 
Hi Henrik

Here's an interesting bug.
Or at least I think it's a bug.
I set the MAXMSG_CLIENT to 1024, and it fixed the problem I was having.
(See below)

Later on, I had to restart the server, and it crapped out.
I remember that sometimes deleting ~hobbit/server/tmp/* before a startup
can fix things, so I gave it a try.
Still no go.
I then restarted it with --debug, and the only log file getting written
to was hobbitlaunch.log (See extract below)

I then reversed the only thing that had changed - removed the
MAXMSG_CLIENT entry, and it all started up fine.

A bug? Or a PEBKAC?

Regards
   Vernon

--- snip ---
2006-08-07 16:23:54 Loading hostnames
2006-08-07 16:23:54 Opening file
/export/home/hobbit//server/etc/bb-hosts
2006-08-07 16:23:54 Opening file
/export/home/hobbit//server/etc/client-local.cfg
2006-08-07 16:23:54 Loading saved state
2006-08-07 16:23:54 Cannot access checkpoint file
/export/home/hobbit//server/tmp/hobbitd.chk for restore
2006-08-07 16:23:54 Setting up network listener on 0.0.0.0:1984
2006-08-07 16:23:54 Setting up signal handlers
2006-08-07 16:23:54 Setting up hobbitd channels
2006-08-07 16:23:54 Setting up status channel (id=1)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',1)
2006-08-07 16:23:54 ftok() returns: 0x100748F
2006-08-07 16:23:54 shmget() returns: 0x1200001E
2006-08-07 16:23:54 Setting up stachg channel (id=2)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',2)
2006-08-07 16:23:54 ftok() returns: 0x200748F
2006-08-07 16:23:54 shmget() returns: 0x1200001F
2006-08-07 16:23:54 Setting up page channel (id=3)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',3)
2006-08-07 16:23:54 ftok() returns: 0x300748F
2006-08-07 16:23:54 shmget() returns: 0x12000020
2006-08-07 16:23:54 Setting up data channel (id=4)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',4)
2006-08-07 16:23:54 ftok() returns: 0x400748F
2006-08-07 16:23:54 shmget() returns: 0x12000021
2006-08-07 16:23:54 Setting up notes channel (id=5)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',5)
2006-08-07 16:23:54 ftok() returns: 0x500748F
2006-08-07 16:23:54 shmget() returns: 0x12000022
2006-08-07 16:23:54 Setting up enadis channel (id=6)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',6)
2006-08-07 16:23:54 ftok() returns: 0x600748F
2006-08-07 16:23:54 shmget() returns: 0x12000023
2006-08-07 16:23:54 Setting up client channel (id=7)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',7)
2006-08-07 16:23:54 ftok() returns: 0x700748F
2006-08-07 16:23:54 shmget() returns: 0x12000024
2006-08-07 16:23:54 Setting up clichg channel (id=8)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',8)
2006-08-07 16:23:54 ftok() returns: 0x800748F
2006-08-07 16:23:54 Could not get shm of size 1048576: Invalid argument
2006-08-07 16:23:54 Cannot setup clichg channel
--- snip ---
▸ quoted from Vernon Everett


-----Original Message-----
From: Everett, Vernon [mailto:user-36f80bd657a9@xymon.invalid] 
Sent: Monday, 7 August 2006 12:17 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] Message to long

Yep, that was it.
Thanks Lars

Cheers
   Vernon 

-----Original Message-----
From: lars ebeling [mailto:user-1fecd3eafd52@xymon.invalid]
Sent: Monday, 7 August 2006 11:53 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Message to long

Have a look at hobbitserver.cfg manpage. MAXMSG_CLIENT

Regards
Lars

----- Original Message -----
From: "Everett, Vernon" <user-36f80bd657a9@xymon.invalid>
To: <user-ae9b8668bcde@xymon.invalid>
Sent: Monday, August 07, 2006 3:42 AM
Subject: [hobbit] Message to long


Hi all

Despite my best efforts, my procs page remains purple.
I found this in the hobbitd page - which also remains yellow.
(Server name changed to pluto)

--- snip ---
Latest errormessages:
Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit
524288)
First line: pluto|sunos|sunos
Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447,
limit=262144)
Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated
(n=508456, limit=262144) Oversize status msg from 10.1.5.242 for
pluto:ports truncated (n=508452,
limit=262144)
Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit
524288)
First line: pluto|sunos|sunos
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448,
limit=262144)
--- snip ---

I am assuming that the output from the ports and procs test is too
large.
How do we make Hobbit happy with the larger message?
This machine is a proxy server, so we can expect many procs and a long
ports output.

Regards
  Vernon
list Colin Spargo · Mon, 7 Aug 2006 09:48:24 +0100 · 
MAXMSG_CLIENT seems to need to allocate a shared memory segment of equal 
size. If your kernel shared memory settings are set too low (eg. if you 
are running defaults on solaris) then you may not be able to allocate the 
segment. If that happens, things like you just described seem to happen... 
 

you could possibly have dangling segments left over that were not cleaned 
up when you restarted hobbit, or possibly another app.

Try checking what is allocated with "ipcs -a", and increasing the 
parameter "shmmax".

This is a PRIVATE message. If you are not the intended recipient, please 
delete without copying and kindly advise us by e-mail of the mistake in 
delivery. NOTE: Regardless of content, this e-mail shall not operate to 
bind CSC to any order or other contract unless pursuant to explicit 
written agreement or government initiative expressly permitting the use of 
e-mail for such purpose.


"Everett, Vernon" <user-36f80bd657a9@xymon.invalid> 
07/08/2006 09:40
Please respond to
user-ae9b8668bcde@xymon.invalid


To
<user-ae9b8668bcde@xymon.invalid>
cc
▸ quoted from Vernon Everett

Subject
RE: [hobbit] Message to long


Hi Henrik

Here's an interesting bug.
Or at least I think it's a bug.
I set the MAXMSG_CLIENT to 1024, and it fixed the problem I was having.
(See below)

Later on, I had to restart the server, and it crapped out.
I remember that sometimes deleting ~hobbit/server/tmp/* before a startup
can fix things, so I gave it a try.
Still no go.
I then restarted it with --debug, and the only log file getting written
to was hobbitlaunch.log (See extract below)

I then reversed the only thing that had changed - removed the
MAXMSG_CLIENT entry, and it all started up fine.

A bug? Or a PEBKAC?

Regards
   Vernon

--- snip ---
2006-08-07 16:23:54 Loading hostnames
2006-08-07 16:23:54 Opening file
/export/home/hobbit//server/etc/bb-hosts
2006-08-07 16:23:54 Opening file
/export/home/hobbit//server/etc/client-local.cfg
2006-08-07 16:23:54 Loading saved state
2006-08-07 16:23:54 Cannot access checkpoint file
/export/home/hobbit//server/tmp/hobbitd.chk for restore
2006-08-07 16:23:54 Setting up network listener on 0.0.0.0:1984
2006-08-07 16:23:54 Setting up signal handlers
2006-08-07 16:23:54 Setting up hobbitd channels
2006-08-07 16:23:54 Setting up status channel (id=1)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',1)
2006-08-07 16:23:54 ftok() returns: 0x100748F
2006-08-07 16:23:54 shmget() returns: 0x1200001E
2006-08-07 16:23:54 Setting up stachg channel (id=2)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',2)
2006-08-07 16:23:54 ftok() returns: 0x200748F
2006-08-07 16:23:54 shmget() returns: 0x1200001F
2006-08-07 16:23:54 Setting up page channel (id=3)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',3)
2006-08-07 16:23:54 ftok() returns: 0x300748F
2006-08-07 16:23:54 shmget() returns: 0x12000020
2006-08-07 16:23:54 Setting up data channel (id=4)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',4)
2006-08-07 16:23:54 ftok() returns: 0x400748F
2006-08-07 16:23:54 shmget() returns: 0x12000021
2006-08-07 16:23:54 Setting up notes channel (id=5)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',5)
2006-08-07 16:23:54 ftok() returns: 0x500748F
2006-08-07 16:23:54 shmget() returns: 0x12000022
2006-08-07 16:23:54 Setting up enadis channel (id=6)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',6)
2006-08-07 16:23:54 ftok() returns: 0x600748F
2006-08-07 16:23:54 shmget() returns: 0x12000023
2006-08-07 16:23:54 Setting up client channel (id=7)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',7)
2006-08-07 16:23:54 ftok() returns: 0x700748F
2006-08-07 16:23:54 shmget() returns: 0x12000024
2006-08-07 16:23:54 Setting up clichg channel (id=8)
2006-08-07 16:23:54 calling ftok('/export/home/hobbit//server',8)
2006-08-07 16:23:54 ftok() returns: 0x800748F
2006-08-07 16:23:54 Could not get shm of size 1048576: Invalid argument
2006-08-07 16:23:54 Cannot setup clichg channel
--- snip ---


-----Original Message-----
From: Everett, Vernon [mailto:user-36f80bd657a9@xymon.invalid] 
Sent: Monday, 7 August 2006 12:17 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] Message to long

Yep, that was it.
Thanks Lars

Cheers
   Vernon 

-----Original Message-----
From: lars ebeling [mailto:user-1fecd3eafd52@xymon.invalid]
Sent: Monday, 7 August 2006 11:53 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Message to long

Have a look at hobbitserver.cfg manpage. MAXMSG_CLIENT

Regards
Lars

----- Original Message -----
From: "Everett, Vernon" <user-36f80bd657a9@xymon.invalid>
To: <user-ae9b8668bcde@xymon.invalid>
Sent: Monday, August 07, 2006 3:42 AM
Subject: [hobbit] Message to long


Hi all

Despite my best efforts, my procs page remains purple.
I found this in the hobbitd page - which also remains yellow.
(Server name changed to pluto)

--- snip ---
Latest errormessages:
Oversize data/client msg from 10.1.5.242 truncated (n=949733, limit
524288)
First line: pluto|sunos|sunos
Oversize stachg msg from 10.1.5.242 for pluto:ports truncated (n=508447,
limit=262144)
Oversize page/ack/notify msg from 10.1.5.242 for pluto:ports truncated
(n=508456, limit=262144) Oversize status msg from 10.1.5.242 for
pluto:ports truncated (n=508452,
limit=262144)
Oversize data/client msg from 10.1.5.242 truncated (n=977049, limit
524288)
First line: pluto|sunos|sunos
Oversize status msg from 10.1.5.242 for pluto:ports truncated (n=508448,
limit=262144)
--- snip ---

I am assuming that the output from the ports and procs test is too
large.
How do we make Hobbit happy with the larger message?
This machine is a proxy server, so we can expect many procs and a long
ports output.

Regards
  Vernon
list Vernon Everett · Mon, 7 Aug 2006 17:47:04 +0800 · 
On Solaris 10?
▸ quoted from Colin Spargo
 

From: Colin Spargo [mailto:user-4148d5b43ace@xymon.invalid] 
Sent: Monday, 7 August 2006 4:48 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] Message to long


MAXMSG_CLIENT seems to need to allocate a shared memory segment of equal
size. If your kernel shared memory settings are set too low (eg. if you
are running defaults on solaris) then you may not be able to allocate
the segment. If that happens, things like you just described seem to
happen...   

you could possibly have dangling segments left over that were not
cleaned up when you restarted hobbit, or possibly another app.

Try checking what is allocated with "ipcs -a", and increasing the
parameter "shmmax". 

This is a PRIVATE message. If you are not the intended recipient, please
delete without copying and kindly advise us by e-mail of the mistake in
delivery. NOTE: Regardless of content, this e-mail shall not operate to
bind CSC to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use
of e-mail for such purpose.
list Colin Spargo · Mon, 7 Aug 2006 11:07:59 +0100 · 
Not as likely but still possible. I think the default limit is 25% of physmem, so it depends. If you have leftover segments or a small amount of physical memory then this may be the cause:

The error message that you got in the log definitely points to this:

"2006-08-07 16:23:54 Could not get shm of size 1048576: Invalid argument"

"Invalid argument" means:

"     EINVAL          The size argument is less than  the  system-
                     imposed  minimum or greater than the system-
                     imposed maximum. See NOTES.

                     A shared memory identifier  exists  for  key
                     but  the size of the segment associated with
                     it is less than size and size is  not  equal
                     to 0.
"


"Everett, Vernon" <user-36f80bd657a9@xymon.invalid> 07/08/2006 10:47
▸ quoted from Vernon Everett
Please respond to
user-ae9b8668bcde@xymon.invalid


To
<user-ae9b8668bcde@xymon.invalid>
cc

Subject
RE: [hobbit] Message to long


On Solaris 10?
 
From: Colin Spargo [mailto:user-4148d5b43ace@xymon.invalid] Sent: Monday, 7 August 2006 4:48 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] Message to long


MAXMSG_CLIENT seems to need to allocate a shared memory segment of equal size. If your kernel shared memory settings are set too low (eg. if you are running defaults on solaris) then you may not be able to allocate the segment. If that happens, things like you just described seem to happen...   
you could possibly have dangling segments left over that were not cleaned up when you restarted hobbit, or possibly another app.

Try checking what is allocated with "ipcs -a", and increasing the parameter "shmmax".