Reversing the effects of a 'bb 127.0.0.1 "drop HOSTNAME TEST"' command
list Jonathan B. Horen
!@#$% I've never been good a dealing with the aftermath of purple alerts, and always end-up using brute-force to remove all hostname references in $XYMON/data. But this time I really screwed-up and ran bb 127.0.0.1 "drop HOSTNAME TEST" for the purple-affected hosts... and now I can't get 'em back! I need help/suggestions/advice TIA! -- JONATHAN B. HOREN ARSC/LSI Systems Administrator WRRB/008-001 T: (XXX) XXX-XXXX E: user-12d4882938ba@xymon.invalid *"After Tuesday, even the calendar says W T F!!"*
list Buchan Milne
▸
----- "Jonathan B. Horen" <user-12d4882938ba@xymon.invalid> wrote:
!@#$% I've never been good a dealing with the aftermath of purple alerts,
Why do you get so many purple alerts, and why is there an "aftermath" ?
▸
and always end-up using brute-force to remove all hostname references in $XYMON/data.
Why?
▸
But this time I really screwed-up and ran bb 127.0.0.1 "drop HOSTNAME TEST" for the purple-affected hosts... and now I can't get 'em back!
They were purple because they were not being reported (e.g. the client was running, or running with wrong host name, or bbnet was not testing them for some reason). Getting them back (to show on the display) takes the same steps as resolving the original purple problem, get them to be reported on ...
I need help/suggestions/advice
If you want the historical data too, restore from backup (data/hist/${MACHINE}.${test} and data/histlogs/${MACHINEDOTS}/${test})
Regards,
Buchan
list Jonathan B. Horen
▸
On Wed, Mar 2, 2011 at 4:21 PM, Ralph Mitchell <user-00a5e44c48c0@xymon.invalid>wrote:
On Wed, Mar 2, 2011 at 7:37 PM, Jonathan B. Horen <user-12d4882938ba@xymon.invalid>wrote:!@#$% I've never been good a dealing with the aftermath of purple alerts, and always end-up using brute-force to remove all hostname references in $XYMON/data. But this time I really screwed-up and ran bb 127.0.0.1 "drop HOSTNAME TEST" for the purple-affected hosts... and now I can't get 'em back!
I could be wrong, but I think you're pretty much done, as far as the old
information goes. Actually, I'm not sure about the RRDs. However, whatever
was purple will come back just as soon as the next report is delivered. If
you want to put the dots back up manually, try this:
bb 127.0.0.1 "status hostname,domain,com.stuff green"
to send a green status for the "stuff" column. Those are commas in the
hostname, because back in the day BB used the period to separate the test
name from the host name. Give it 30 mins and it'll go purple all on its
own... :)
You could follow up with:
bb 127.0.0.1 "disable hostname,domain,com.stuff -1 wups, broke it"
to send "disable until OK", so those pesky purple dots stay away.
I guess I didn't express myself correctly. I need the tests to restart, not
to have a green status-LED place-holder.
At this point, I've:
- stopped xymon on the server
- stopped xymon on the three affected clients
- copied all of $XYMON/data/{hist,hostdata,hostlogs,rrd} from my last
backup to $XYMON/data
- started xymon on the server
- started xymon on the three affected clients
No joy. The LEDs for cpu, disk, files, memory, msgs, ports, and procs on the
three affected hosts remain missing.
On each of the affected clients,
$XYMON/tmp/{hobbit_vmstat.biotech.nnn,msg.biotech.txt) exist and are updated
regularly (@5-minute intervals).
So, what's wrong on the server? Why isn't the information being
generated-and-displayed on bb.html?
What am I missing?
▸
--
JONATHAN B. HOREN ARSC/LSI Systems Administrator
WRRB/008-001 T: (XXX) XXX-XXXX E: user-12d4882938ba@xymon.invalid
*"After Tuesday, even the calendar says W T F!!"*
list Jonathan B. Horen
▸
On Wed, Mar 2, 2011 at 3:37 PM, Jonathan B. Horen <user-12d4882938ba@xymon.invalid>wrote:
!@#$% I've never been good a dealing with the aftermath of purple alerts, and always end-up using brute-force to remove all hostname references in $XYMON/data. But this time I really screwed-up and ran bb 127.0.0.1 "drop HOSTNAME TEST" for the purple-affected hosts... and now I can't get 'em back! I need help/suggestions/advice TIA!
Forget any mention of "purple". How do I reverse the effects of the aforementioned command? In other words, how do I undo a "dropped" test?
list Henrik Størner
▸
Den 03-03-2011 18:46, Jonathan B. Horen skrev:
I guess I didn't express myself correctly. I need the tests to restart,
not to have a green status-LED place-holder.
At this point, I've:
* stopped xymon on the server
* stopped xymon on the three affected clients
* copied all of $XYMON/data/{hist,hostdata,hostlogs,rrd} from my
last backup to $XYMON/data
* started xymon on the server
* started xymon on the three affected clients
No joy. The LEDs for cpu, disk, files, memory, msgs, ports, and procs on
the three affected hosts remain missing.Your clients are not sending data to the Xymon server. If they did, then the cpu, disk etc. statuses would show up right away.
▸
On each of the affected clients,
$XYMON/tmp/{hobbit_vmstat.biotech.nnn,msg.biotech.txt) exist and are
updated regularly (@5-minute intervals).
So, what's wrong on the server? Why isn't the information being
generated-and-displayed on bb.html?Check the "Reports->Ghost clients" display to see if the hosts are reporting with another name than what's in the hosts.cfg / bb-hosts file. And check the logfiles on the clients to see if there are problems sending data over to the Xymon server. Regards, Henrik
list Ryan Novosielski
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/03/2011 01:54 PM, Jonathan B. Horen wrote:
On Wed, Mar 2, 2011 at 3:37 PM, Jonathan B. Horen <user-12d4882938ba@xymon.invalid
▸
<mailto:user-12d4882938ba@xymon.invalid>> wrote:
!@#$%
I've never been good a dealing with the aftermath of purple alerts,
and always end-up using brute-force to remove all hostname
references in $XYMON/data. But this time I really screwed-up and ran
bb 127.0.0.1 "drop HOSTNAME TEST"
for the purple-affected hosts... and now I can't get 'em back!
I need help/suggestions/advice
TIA!
Forget any mention of "purple".
How do I reverse the effects of the aforementioned command? In other
words, how do I undo a "dropped" test?With a good backup, if I'm not mistaken. I believe the drop command gets all of the data removed from disk. - -- - ---- _ _ _ _ ___ _ _ _ |Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Sr. Systems Programmer |$&| |__| | | |__/ | \| _| |user-ae4522577e16@xymon.invalid - 973/972.0922 (2-0922) \__/ Univ. of Med. and Dent.|IST/CST-Academic Svcs. - ADMC 450, Newark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1wUK8ACgkQmb+gadEcsb4W1gCgzERajZECxb5gchWpdbQ9zngL T1MAn3wUSPUKBumlayCMntnqdnwg3QmN =w06E -----END PGP SIGNATURE-----
list Daniel Nordquist
I believe you're correct. A drop will remove all histlogs for that host/test.
▸
-----Original Message-----
From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Ryan Novosielski
Sent: Thursday, March 03, 2011 9:39 PM
To: xymon at xymon.com
Subject: Re: [Xymon] Reversing the effects of a 'bb 127.0.0.1 "drop HOSTNAME TEST"' command
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/03/2011 01:54 PM, Jonathan B. Horen wrote:
On Wed, Mar 2, 2011 at 3:37 PM, Jonathan B. Horen <user-12d4882938ba@xymon.invalid
<mailto:user-12d4882938ba@xymon.invalid>> wrote:
!@#$%
I've never been good a dealing with the aftermath of purple alerts,
and always end-up using brute-force to remove all hostname
references in $XYMON/data. But this time I really screwed-up and
ran
bb 127.0.0.1 "drop HOSTNAME TEST"
for the purple-affected hosts... and now I can't get 'em back!
I need help/suggestions/advice
TIA!
Forget any mention of "purple".
How do I reverse the effects of the aforementioned command? In other
words, how do I undo a "dropped" test?With a good backup, if I'm not mistaken. I believe the drop command gets all of the data removed from disk. - -- - ---- _ _ _ _ ___ _ _ _ |Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Sr. Systems Programmer |$&| |__| | | |__/ | \| _| |user-ae4522577e16@xymon.invalid - 973/972.0922 (2-0922) \__/ Univ. of Med. and Dent.|IST/CST-Academic Svcs. - ADMC 450, Newark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1wUK8ACgkQmb+gadEcsb4W1gCgzERajZECxb5gchWpdbQ9zngL T1MAn3wUSPUKBumlayCMntnqdnwg3QmN =w06E -----END PGP SIGNATURE-----
This e-mail message and any attached files are confidential and are intended solely for the use of the addressee(s) named above. If you are not the intended recipient, any review, use, or distribution of this e-mail message and any attached files is strictly prohibited.
This communication may contain material protected by Federal privacy regulations, attorney-client work product, or other privileges. If you have received this confidential communication in error, please notify the sender immediately by reply e-mail message and permanently delete the original message. To reply to our email administrator directly, send an email to: user-40803320aaf5@xymon.invalid .
If this e-mail message concerns a contract matter, be advised that no employee or agent is authorized to conclude any binding agreement on behalf of Orlando Health by e-mail without express written confirmation by an officer of the corporation. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Orlando Health.