Where to find the latest hostdata (or: How to monitor one process on two servers)?
list Martin Ward
Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can't have something that runs on the client since I won't know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/|
[Colt Disclaimer]
The message is intended for the named addressee only and may not be disclosed
to or used by anyone else, nor may it be copied in any way. The contents of
this message and its attachments are confidential and may also be subject to
legal privilege. If you are not the named addressee and/or have received this
message in error, please advise us by e-mailing user-51905b889b93@xymon.invalid and delete the
message and any attachments without retaining any copies. Internet
communications are not secure and Colt does not accept responsibility for this
message, its contents nor responsibility for any viruses. No contracts can be
created or varied on behalf of Colt Technology Services, its subsidiaries,
group companies or affiliates ("Colt") and any other party by email
communications unless expressly agreed in writing with such other party.
Please note that incoming emails will be automatically scanned to eliminate
potential viruses and unsolicited promotional emails. For more information
refer to www.colt.net or contact us on +44(0)20 7390 3900
list Steven Carr
You can get this using the xymon command e.g... xymon $XYMSERVERS "clientlog hostname section=ps" That will dump the [ps] section from the current client data, you can then grep through that to look for the required process, repeat for the second host and then hopefully you should know which one it's running on. Steve
▸
On 12 January 2012 16:05, Ward, Martin <user-2d33a6eb6a05@xymon.invalid> wrote:
Hi all,****
** **
I have a pair of servers that run Veritas Cluster Software and they have a
number of different processes that they run in turn. Simply put it means
that there is a process that will be running on one or another of the two
servers and at any given time I will not know which server it runs on. The
important thing for me is that it is running, so I set about creating a
monitor that will check for this.****
** **
I can’t have something that runs on the client since I won’t know which
client the process runs on but it struck me that the server always has an
up to date process list, so I could simply read the hostdata file for the
two servers, rip out the process list, concatenate the two and search for
the specific process name, simple!****
** **
This is a brilliant idea that would work if only I had access to the up to
date client data. It seems that the data stored in the hostdata directory
only changes when a status for that server changes to an alert state
(according to the xymond_hostdata web page) so if the process switches to a
different machine but nothing actually changes the alert status of the
client, the latest host data is not stored in the hostdata/ subdirectory.*
***
** **
I know that the latest host data is stored somewhere because I can see it
in the web browser. Does anyone know where this up to date host data is
kept such that I can utilise the information in it?****
** **
Failing that do you have any other suggestions, recommendations or scripts
lying around that will enable me to look for a process across more than one
server?****
** **
|\/|****
[Colt Disclaimer] The message is intended for the named addressee only
and may not be disclosed to or used by anyone else, nor may it be copied in
any way. The contents of this message and its attachments are confidential
and may also be subject to legal privilege. If you are not the named
addressee and/or have received this message in error, please advise us by
e-mailing user-51905b889b93@xymon.invalid and delete the message and any attachments
without retaining any copies. Internet communications are not secure and
Colt does not accept responsibility for this message, its contents nor
responsibility for any viruses. No contracts can be created or varied on
behalf of Colt Technology Services, its subsidiaries, group companies or
affiliates ("Colt") and any other party by email communications unless
expressly agreed in writing with such other party. Please note that
incoming emails will be automatically scanned to eliminate potential
viruses and unsolicited promotional emails. For more information refer to
www.colt.net or contact us on +44(0)20 7390 3900
list Asif Iqbal
▸
On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <user-2d33a6eb6a05@xymon.invalid> wrote:
Hi all, I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this. I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple! This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c will let you know if the process is running on client.example.com create a new column for that process called proc for all your hosts with an ext script on the server Then use combo.cfg to generate alert like this allhosts.proc = ( client1.proc + client2.gis ) == 1 as long as the result is 1 allhosts.proc won't be red HTH
▸
I know that the latest host data is stored somewhere because I can see it in
the web browser. Does anyone know where this up to date host data is kept
such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts
lying around that will enable me to look for a process across more than one
server?
|\/|
[Colt Disclaimer] The message is intended for the named addressee only and
may not be disclosed to or used by anyone else, nor may it be copied in any
way. The contents of this message and its attachments are confidential and
may also be subject to legal privilege. If you are not the named addressee
and/or have received this message in error, please advise us by e-mailing
user-51905b889b93@xymon.invalid and delete the message and any attachments without retaining
any copies. Internet communications are not secure and Colt does not accept
responsibility for this message, its contents nor responsibility for any
viruses. No contracts can be created or varied on behalf of Colt Technology
Services, its subsidiaries, group companies or affiliates ("Colt") and any
other party by email communications unless expressly agreed in writing with
such other party. Please note that incoming emails will be automatically
scanned to eliminate potential viruses and unsolicited promotional emails.
For more information refer to www.colt.net or contact us on +44(0)20 7390
3900
--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
list Asif Iqbal
▸
On Thu, Jan 12, 2012 at 12:13 PM, Asif Iqbal <user-6f4b51ac2a40@xymon.invalid> wrote:
On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <user-2d33a6eb6a05@xymon.invalid> wrote:Hi all, I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this. I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple! This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c will let you know if the process is running on client.example.com create a new column for that process called proc for all your hosts with an ext script on the server Then use combo.cfg to generate alert like this allhosts.proc = ( client1.proc + client2.gis ) == 1
correction: allhosts.proc = (client1.proc + client2.proc) == 1
▸
as long as the result is 1 allhosts.proc won't be red HTHI know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it? Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server? |\/| [Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing user-51905b889b93@xymon.invalid and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
list Bruce White
Actually, I have multiple HP Service Guard clusters which have the same basic issue you listed for your VCS cluster. I solved the issue by copying the Xymon client directory on the cluster node and naming it something which matched the "package" name (or Service Group in VCS), editing down the xymonclient-hp-ux.sh to just date, df, mount, and ps sections. I added the stop/start of this new client process into the Service Guard Package start-up script (VCS service Group stop/start scripts). With careful ignores in the analysis.cfg (was hobbit-clients.cfg) file, I control what disk space is monitored by which client process. I then added an entry in the hosts.cfg (was bb-hosts) file with the IP associated with the Package (Service Group). No matter where the Package (SG) is running on the cluster, the info is passed back to Xymon. If a Package is not running, then the client stops. Of course if the Package is down for any time, Xymon could turn the "conn" dot red and send out alerts. I can also monitor at the new package client level any processes particular to that package and not worry what server is actually running the package. If they stop, the package alerts, not the node in the cluster. I also added a custom script to monitor the output of the cmviewcl command (cluster status similar to the hastatus -summary command in VCS). If the output indicates a failure (package or node) then this test (running on each node in the cluster) goes red and I get an alert that something on the cluster is not good. Hope that helps, Bruce Bruce White Senior Enterprise Systems Engineer | Phone: X-XXX-XXX-XXXX | Fax: XXX-XXX-XXXX | user-58f975e8bf9d@xymon.invalid | http://www.fellowes.com/ Disclaimer: The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. Fellowes, Inc. -----Original Message----- From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Asif Iqbal Sent: Thursday, January 12, 2012 11:14 AM To: Ward, Martin Cc: xymon at xymon.com Subject: Re: [Xymon] Where to find the latest hostdata (or: How to monitor one process on two servers)?
▸
On Thu, Jan 12, 2012 at 12:13 PM, Asif Iqbal <user-6f4b51ac2a40@xymon.invalid> wrote:On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <user-2d33a6eb6a05@xymon.invalid> wrote:Hi all, I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this. I can't have something that runs on the client since I won't know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple! This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c will let you know if the process is running on client.example.com create a new column for that process called proc for all your hosts with an ext script on the server Then use combo.cfg to generate alert like this allhosts.proc = ( client1.proc + client2.gis ) == 1
correction: allhosts.proc = (client1.proc + client2.proc) == 1
as long as the result is 1 allhosts.proc won't be red HTHI know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it? Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server? |\/| [Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing user-51905b889b93@xymon.invalid and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt
Technology
Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20
7390
3900-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
list Asif Iqbal
▸
On Thu, Jan 12, 2012 at 12:14 PM, Asif Iqbal <user-6f4b51ac2a40@xymon.invalid> wrote:
On Thu, Jan 12, 2012 at 12:13 PM, Asif Iqbal <user-6f4b51ac2a40@xymon.invalid> wrote:On Thu, Jan 12, 2012 at 11:05 AM, Ward, Martin <user-2d33a6eb6a05@xymon.invalid> wrote:Hi all, I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this. I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple! This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.xymon localhost "clientlog client.example.com section=ps" | grep pr[o]c will let you know if the process is running on client.example.com create a new column for that process called proc for all your hosts with an ext script on the server Then use combo.cfg to generate alert like this allhosts.proc = ( client1.proc + client2.gis ) == 1correction: allhosts.proc = (client1.proc + client2.proc) == 1
forgot to mention the last part. you will also need to have an entry on hosts.cfg, something like this page allhosts All Hosts proc service group-compress <H3><I>All Hosts proc service</I></H3> group-only proc|info 0.0.0.0 allhosts # noconn NAME:"Combo test for proc on all hosts"
▸
as long as the result is 1 allhosts.proc won't be red HTHI know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it? Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server? |\/| [Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing user-51905b889b93@xymon.invalid and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
list Martin Ward
My thanks to Steven, Bruce and Iqbal for your suggestions, with your help I finally managed to get a nice, simple script that runs on the server, checks the process list from both clients and creates a Xymon message for a dummy host that sets the status of the process. In this instance it was simpler and cleaner to have a single script running on the server than to have client-side scripts plus combo.cfg entries, ymmv. |\/|artin
▸
From: Steven Carr [mailto:user-923b20c0d620@xymon.invalid]
Sent: 12 January 2012 17:09
To: Ward, Martin
Cc: xymon at xymon.com
Subject: Re: [Xymon] Where to find the latest hostdata (or: How to monitor one process on two servers)?
You can get this using the xymon command e.g...
xymon $XYMSERVERS "clientlog hostname section=ps"
That will dump the [ps] section from the current client data, you can then grep through that to look for the required process, repeat for the second host and then hopefully you should know which one it's running on.
Steve
On 12 January 2012 16:05, Ward, Martin <user-2d33a6eb6a05@xymon.invalid<mailto:user-2d33a6eb6a05@xymon.invalid>> wrote:
Hi all,
I have a pair of servers that run Veritas Cluster Software and they have a number of different processes that they run in turn. Simply put it means that there is a process that will be running on one or another of the two servers and at any given time I will not know which server it runs on. The important thing for me is that it is running, so I set about creating a monitor that will check for this.
I can’t have something that runs on the client since I won’t know which client the process runs on but it struck me that the server always has an up to date process list, so I could simply read the hostdata file for the two servers, rip out the process list, concatenate the two and search for the specific process name, simple!
This is a brilliant idea that would work if only I had access to the up to date client data. It seems that the data stored in the hostdata directory only changes when a status for that server changes to an alert state (according to the xymond_hostdata web page) so if the process switches to a different machine but nothing actually changes the alert status of the client, the latest host data is not stored in the hostdata/ subdirectory.
I know that the latest host data is stored somewhere because I can see it in the web browser. Does anyone know where this up to date host data is kept such that I can utilise the information in it?
Failing that do you have any other suggestions, recommendations or scripts lying around that will enable me to look for a process across more than one server?
|\/|[Colt Disclaimer] The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing user-51905b889b93@xymon.invalid<mailto:user-51905b889b93@xymon.invalid> and delete the message and any attachments without retaining any copies. Internet communications are not secure and Colt does not accept responsibility for this message, its contents nor responsibility for any viruses. No contracts can be created or varied on behalf of Colt Technology Services, its subsidiaries, group companies or affiliates ("Colt") and any other party by email communications unless expressly agreed in writing with such other party. Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net<http://www.colt.net>; or contact us on +44(0)20 7390 3900<tel:%2B44%280%2920%207390%203900>