hobbitfetch replacement with ssh
list Dave Haertig
I'm still thinking about my hobbitfetch/msgcache replacement I asked about a few weeks ago. Using only ssh from the Hobbit server. Client machine is out in DMZ, thus unable to talk to Hobbit server. I cannot run a listener (msgcache) on the client (policy reasons, not technical ones). Here are my thoughts. Maybe someone can expand on them or suggest a different approach. (1) Run normal Hobbit client on DMZ machine. (1a) Client appears to collect data just fine into ~hobbit/tmp/msg.XXX.txt (1b) Client would try to connect to Hobbit server to transmit this file, and fail. Fine. Let it fail. It appears to do so gracefully. Question: Any simple way to make it stop trying to connect and still behave gracefully? chmod 000 ~hobbit/bin/bb? Change BBDISP="" in ~hobbit/etc/hobbitclient.cfg? (2) Create a Hobbit server script to use socks/ssh to retrieve that ~hobbit/tmp/msg.XXX.txt file from the client machine (it's trivial to write such a thing) (3) Have this server script feed the raw data retrieved in step (2) into the Hobbit listener (which would be localhost:1984), such that it appears to the Hobbit listener the data came in from the client computer in the normal manner. Step (3) is what I don't quite know how to implement yet. I'm familiar with using the bb command to send messages, but those are formatted messages, not raw data as would be present in the retrieved msg.XXX.tmp file. I want to assure that the guts of the Hobbit server processes this proxied client raw data just as if the client had sent it instead of my custom server script. Is there some trivial way to accomplish (3) that I'm just missing searching through the man pages? I could have my custom server script parse the raw data itself and then create individual Hobbit messages (to send via the bb command), but that sounds like I'd be reinventing the wheel. Hobbit server processes already know how to parse this data ... if I could just tap into that part of it. Thanks!
list Dave Haertig
Oops. Looks like I missed the obvious (again!). bb accepts a message of type "client". What do you bet I've answered my own question here? (again!) I think I'll try a "client" message, and I bet I'll be pleasantly surprized. And if anyone wants to know the answer to the question running around in your minds ... YES, it does hurt to be this stupid! Ouch! ;-) Thanks
▸
From: Haertig, David F (Dave) [mailto:user-68874b735d77@xymon.invalid]
Sent: Friday, June 22, 2007 7:27 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] hobbitfetch replacement with ssh
I'm still thinking about my hobbitfetch/msgcache replacement I asked
about a few weeks ago. Using only ssh from the Hobbit server. Client
machine is out in DMZ, thus unable to talk to Hobbit server. I cannot
run a listener (msgcache) on the client (policy reasons, not technical
ones).
Here are my thoughts. Maybe someone can expand on them or suggest a
different approach.
(1) Run normal Hobbit client on DMZ machine.
(1a) Client appears to collect data just fine into
~hobbit/tmp/msg.XXX.txt
(1b) Client would try to connect to Hobbit server to transmit this file,
and fail. Fine. Let it fail. It appears to do so gracefully.
Question: Any simple way to make it stop trying to connect and still
behave gracefully? chmod 000 ~hobbit/bin/bb? Change BBDISP="" in
~hobbit/etc/hobbitclient.cfg?
(2) Create a Hobbit server script to use socks/ssh to retrieve that
~hobbit/tmp/msg.XXX.txt file from the client machine (it's trivial to
write such a thing)
(3) Have this server script feed the raw data retrieved in step (2) into
the Hobbit listener (which would be localhost:1984), such that it
appears to the Hobbit listener the data came in from the client computer
in the normal manner.
Step (3) is what I don't quite know how to implement yet. I'm familiar
with using the bb command to send messages, but those are formatted
messages, not raw data as would be present in the retrieved msg.XXX.tmp
file. I want to assure that the guts of the Hobbit server processes
this proxied client raw data just as if the client had sent it instead
of my custom server script.
Is there some trivial way to accomplish (3) that I'm just missing
searching through the man pages? I could have my custom server script
parse the raw data itself and then create individual Hobbit messages (to
send via the bb command), but that sounds like I'd be reinventing the
wheel. Hobbit server processes already know how to parse this data ...
if I could just tap into that part of it.
Thanks!
list Dave Haertig
Yes, this wasn't terribly hard at all, now was it? I feel dumb.
Server script (still need to beef it up with error checking):
export SOCKS5_USER=xxx
export SOCKS5_PASSWD=xxx
export DMZ_MACH=my.machine.dmz
export TMPFILE=something_or_other
runsocks scp ${DMZ_MACH}:tmp/msg.${DMZ_MACH}.txt ${TMPFILE}
${BB} $(BBDISP} "`cat ${TMPFILE}`"
rm -f ${TMPFILE}
The one thing I appears to be missing doing things like this is that
"[Client data available]" link on the generated webpages. I don't know
why that's missing yet.
And I also need to investigate the possible problems of having the
client script and the server script run asynchronously. It may be
possible to upload the same msg.XXX.txt file twice in a row if
timings/delays fall just right. I need to see how that would affect the
RRD/graphing part of Hobbit. It might confuse the heck out of it.
I'm getting there in my quest for an ssh replacement for hobbitfetch.
But comments and suggestions still greatly appreciated!
▸
Thanks
From: Haertig, David F (Dave) [mailto:user-68874b735d77@xymon.invalid]
Sent: Friday, June 22, 2007 8:11 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] hobbitfetch replacement with ssh
Oops. Looks like I missed the obvious (again!).
bb accepts a message of type "client". What do you bet I've answered my
own question here? (again!) I think I'll try a "client" message, and I
bet I'll be pleasantly surprized.
And if anyone wants to know the answer to the question running around in
your minds ... YES, it does hurt to be this stupid! Ouch! ;-)
Thanks
From: Haertig, David F (Dave) [mailto:user-68874b735d77@xymon.invalid]
Sent: Friday, June 22, 2007 7:27 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] hobbitfetch replacement with ssh
I'm still thinking about my hobbitfetch/msgcache replacement I asked
about a few weeks ago. Using only ssh from the Hobbit server. Client
machine is out in DMZ, thus unable to talk to Hobbit server. I cannot
run a listener (msgcache) on the client (policy reasons, not technical
ones).
Here are my thoughts. Maybe someone can expand on them or suggest a
different approach.
(1) Run normal Hobbit client on DMZ machine.
(1a) Client appears to collect data just fine into
~hobbit/tmp/msg.XXX.txt
(1b) Client would try to connect to Hobbit server to transmit this file,
and fail. Fine. Let it fail. It appears to do so gracefully.
Question: Any simple way to make it stop trying to connect and still
behave gracefully? chmod 000 ~hobbit/bin/bb? Change BBDISP="" in
~hobbit/etc/hobbitclient.cfg?
(2) Create a Hobbit server script to use socks/ssh to retrieve that
~hobbit/tmp/msg.XXX.txt file from the client machine (it's trivial to
write such a thing)
(3) Have this server script feed the raw data retrieved in step (2) into
the Hobbit listener (which would be localhost:1984), such that it
appears to the Hobbit listener the data came in from the client computer
in the normal manner.
Step (3) is what I don't quite know how to implement yet. I'm familiar
with using the bb command to send messages, but those are formatted
messages, not raw data as would be present in the retrieved msg.XXX.tmp
file. I want to assure that the guts of the Hobbit server processes
this proxied client raw data just as if the client had sent it instead
of my custom server script.
Is there some trivial way to accomplish (3) that I'm just missing
searching through the man pages? I could have my custom server script
parse the raw data itself and then create individual Hobbit messages (to
send via the bb command), but that sounds like I'd be reinventing the
wheel. Hobbit server processes already know how to parse this data ...
if I could just tap into that part of it.
Thanks!
list Stef Coene
▸
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:
And I also need to investigate the possible problems of having the client script and the server script run asynchronously. It may be possible to upload the same msg.XXX.txt file twice in a row if timings/delays fall just right. I need to see how that would affect the RRD/graphing part of Hobbit. It might confuse the heck out of it.
Why not ssh to the client, execute hobbit-client.sh and pick up the resulting file ? Stef
list Dave Haertig
Good idea! I just need to see how hobbitlaunch normally passes ENVFILE and LOGFILE to CMD (hobbit-client.sh) when it normally envokes CMD. So I can do that when I invoke hobbit-cient.sh manually via ssh. Probably just as simple env variables. I'll check the source code for hobbitlaunch, or just "guess-and-test". BTW, the clients appear to NOT attempt to connect to Hobbit server, and still behave gracefully, if you set BBDISP=0.0.0.0 and BBDISPLAYS="". At least the logfile quits complaining about that "Whoops!" message. And the msx.XXX.txt file is still generated normally. Great! One last thing is to figure out why there aren't the normal "Client data available" links on the Hobbit webpages (not a show-stopper if they aren't there). Thanks
▸
-----Original Message-----
From: Stef Coene [mailto:user-dbffe946c0f4@xymon.invalid]
Sent: Saturday, June 23, 2007 6:42 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] hobbitfetch replacement with ssh
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:And I also need to investigate the possible problems of having the client script and the server script run asynchronously. It may be possible to upload the same msg.XXX.txt file twice in a row if timings/delays fall just right. I need to see how that would affect the RRD/graphing part of Hobbit. It might confuse the heck out of it.
Why not ssh to the client, execute hobbit-client.sh and pick up the resulting file ? Stef
list Stef Coene
▸
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:
Good idea! I just need to see how hobbitlaunch normally passes ENVFILE and LOGFILE to CMD (hobbit-client.sh) when it normally envokes CMD. So I can do that when I invoke hobbit-cient.sh manually via ssh. Probably just as simple env variables. I'll check the source code for hobbitlaunch, or just "guess-and-test".
You can use bbcmd. So "bbcmd hobbitclient.sh". bbcmd will set the needed environment variabels. You probably have to specify the full path to hobbitclient.sh. Stef
list Dave Haertig
Cool! Thanks for the tip. I was a bit naive in thinking hobbitlaunch might just pass ENVFILE and LOGFILE to CMD as env variables. I had just reviewed hobbitlaunch.c and environ.c and found that hobbitlaunch redirects stdout/stderr to LOGFILE before calling CMD. And also the ENVFILE processing is a little more complex due to the AREA stuff (which implies hobbitclient.cfg is not a sourceable env file, as it almost appears to be in it's default state). I'll look at bbcmd. I was going to hack my own way through this, but if there's already an existing path, I'm all for the easy way!
▸
Thanks
-----Original Message-----
From: Stef Coene [mailto:user-dbffe946c0f4@xymon.invalid]
Sent: Saturday, June 23, 2007 11:55 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] hobbitfetch replacement with ssh
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:Good idea! I just need to see how hobbitlaunch normally passes ENVFILE and LOGFILE to CMD (hobbit-client.sh) when it normally envokes
CMD. So I can do that when I invoke hobbit-cient.sh manually via ssh.
Probably just as simple env variables. I'll check the source code for
hobbitlaunch, or just "guess-and-test".
You can use bbcmd. So "bbcmd hobbitclient.sh". bbcmd will set the needed environment variabels. You probably have to specify the full path to hobbitclient.sh. Stef
list Dave Haertig
Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though. For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally. Second, I could not trick logfetch into grabbing custom logfiles. I tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it. I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client. Thanks
▸
-----Original Message-----
From: Haertig, David F (Dave) [mailto:user-68874b735d77@xymon.invalid]
Sent: Saturday, June 23, 2007 12:20 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] hobbitfetch replacement with ssh
Cool! Thanks for the tip.
I was a bit naive in thinking hobbitlaunch might just pass ENVFILE and
LOGFILE to CMD as env variables. I had just reviewed hobbitlaunch.c and
environ.c and found that hobbitlaunch redirects stdout/stderr to LOGFILE
before calling CMD. And also the ENVFILE processing is a little more
complex due to the AREA stuff (which implies hobbitclient.cfg is not a
sourceable env file, as it almost appears to be in it's default state).
I'll look at bbcmd. I was going to hack my own way through this, but if
there's already an existing path, I'm all for the easy way!
Thanks
-----Original Message-----
From: Stef Coene [mailto:user-dbffe946c0f4@xymon.invalid]
Sent: Saturday, June 23, 2007 11:55 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] hobbitfetch replacement with ssh
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:Good idea! I just need to see how hobbitlaunch normally passes ENVFILE and LOGFILE to CMD (hobbit-client.sh) when it normally envokes
CMD. So I can do that when I invoke hobbit-cient.sh manually via ssh.
Probably just as simple env variables. I'll check the source code for
hobbitlaunch, or just "guess-and-test".
You can use bbcmd. So "bbcmd hobbitclient.sh". bbcmd will set the needed environment variabels. You probably have to specify the full path to hobbitclient.sh. Stef
list Stef Coene
▸
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:
Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though. For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally. Second, I could not trick logfetch into grabbing custom logfiles. I tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it. I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client.
If "I" should have this problem, I will try this: change hobbit client.sh so it: - is not trying to contact the hobbit server (so the bb command is not used) - the message file is saved in a fixed file (so the pid is not appended to it) - the logfetch can still work if you update the input file manually make a daemon on the hobbit server: - that runs every minute - contacts the hobbit client with ssh and checks for a message file, if a message file is found, the content is send to the hobbit server and the file is deleted Or something like this ;) Stef
list John G
Here are some old messages on the topic that are similar to what you are doing. http://www.hswn.dk/hobbiton/2006/01/msg00045.html http://www.hswn.dk/hobbiton/2006/01/msg00047.html
▸
On 6/23/07, Stef Coene <user-dbffe946c0f4@xymon.invalid> wrote:On Saturday 23 June 2007, Haertig, David F (Dave) wrote:Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though. For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally. Second, I could not trick logfetch into grabbing custom logfiles. I tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it. I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client.If "I" should have this problem, I will try this: change hobbit client.sh so it: - is not trying to contact the hobbit server (so the bb command is not used) - the message file is saved in a fixed file (so the pid is not appended to it) - the logfetch can still work if you update the input file manually make a daemon on the hobbit server: - that runs every minute - contacts the hobbit client with ssh and checks for a message file, if a message file is found, the content is send to the hobbit server and the file is deleted Or something like this ;) Stef
list Dave Haertig
Below is what I ended up doing. It's working like a champ. It probably
doesn't fit your environment exactly, but it may be of some help anyway.
My Hobbit server is Linux 2.6, the clients I'm pulling data from are
Solaris 9. All running OpenSSH. The client domains are picked up on
the Hobbit server via /etc/resolv.conf mechanism. Hobbit server has
socks client installed and configured for our environment. ssh public
keys were distributed from host to clients so ssh/scp will work without
passwords being required.
The file "hobbitclient-custom.sh" you see below is nothing more than the
standard hobbitclient.sh file with some stuff I didn't want stripped
out. I modified the LOGFETCHCFG line to point to the "scripts"
directory rather than the default "tmp" directory because I wanted to
keep all my customized stuff in that scripts directory.
Custom HOST script, "pulldata.sh", found on server in $BBHOME/ext
directory, run via clientlaunch.cfg
========================================================================
=============================
export SOCKS5_USER='my_socks_userid'
export SOCKS5_PASSWD='my_socks_password'
USER=stars
for DMZ_MACH in 'machine1' 'machine2' 'machine3'
do
TMPFILE=${BBTMP}/pulldata.${DMZ_MACH}.tmp.$$
runsocks ssh ${USER}@${DMZ_MACH} hobbit/scripts/exec_client.sh
runsocks scp
${USER}@${DMZ_MACH}:~${USER}/hobbit/tmp/msg.${DMZ_MACH}.txt ${TMPFILE}
${BB} ${BBDISP} "`cat ${TMPFILE}`"
rm -f ${TMPFILE}
done
Custom CLIENT script, "exec_client.sh", found on client in
/home/stars/hobbit/scripts directory
========================================================================
=======================
#!/bin/sh
BBHOME=/home/stars/hobbit
BBCMD=${BBHOME}/bin/bbcmd
CMD=${BBHOME}/scripts/hobbitclient-custom.sh
ENVFILE=${BBHOME}/etc/hobbitclient.cfg
LOGFILE=${BBHOME}/logs/hobbitclient-custom.log
${BBCMD} --env=${ENVFILE} ${CMD} >${LOGFILE} 2>&1
exit 0
Custom CLIENT script, "hobbitclient-custom.sh", found on client in
/home/stars/hobbit/scripts directory
========================================================================
===============================
#!/bin/sh
LANG=C
LC_ALL=C
LC_MESSAGES=C
export LANG LC_ALL LC_MESSAGES
if test "$BBOSSCRIPT" = ""; then
BBOSSCRIPT="hobbitclient-`uname -s | tr '[A-Z]' '[a-z]'`.sh"
fi
MSGFILE="$BBTMP/msg.$MACHINEDOTS.txt"
MSGTMPFILE="$MSGFILE.$$"
LOGFETCHCFG=$BBHOME/scripts/logfetch.$MACHINEDOTS.cfg
LOGFETCHSTATUS=$BBTMP/logfetch.$MACHINEDOTS.status
export LOGFETCHCFG LOGFETCHSTATUS
rm -f $MSGTMPFILE
touch $MSGTMPFILE
echo "client $MACHINE.$BBOSTYPE $CONFIGCLASS" >>$MSGTMPFILE
$BBHOME/bin/$BBOSSCRIPT >> $MSGTMPFILE
if test -f $LOGFETCHCFG
then
$BBHOME/bin/logfetch $LOGFETCHCFG $LOGFETCHSTATUS >>$MSGTMPFILE
fi
echo "[clock]" >>$MSGTMPFILE
$BBHOME/bin/logfetch --clock >>$MSGTMPFILE
rm -f $MSGFILE
mv $MSGTMPFILE $MSGFILE
exit 0
Custom CLIENT file, "logfetch.machine1.txt", found on client in
/home/stars/hobbit/scripts directory
========================================================================
============================
log:/my_app/logs/file1:4096
log:/my_app/logs/file2:4096
log:/my_app/logs/file3:4096
▸
-----Original Message----- From: John G [mailto:user-7da77b391823@xymon.invalid] Sent: Monday, June 25, 2007 8:30 PM To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] hobbitfetch replacement with ssh Here are some old messages on the topic that are similar to what you are doing. http://www.hswn.dk/hobbiton/2006/01/msg00045.html http://www.hswn.dk/hobbiton/2006/01/msg00047.html On 6/23/07, Stef Coene <user-dbffe946c0f4@xymon.invalid> wrote:
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though. For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally. Second, I could not trick logfetch into grabbing custom logfiles. I
tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it. I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client.If "I" should have this problem, I will try this: change hobbit client.sh so it: - is not trying to contact the hobbit server (so the bb command is not
used) - the message file is saved in a fixed file (so the pid is not appended to it) - the logfetch can still work if you update the input file manually make a daemon on the hobbit server: - that runs every minute - contacts the hobbit client with ssh and checks for a message file, if a message file is found, the content is send to the hobbit server and the file is deleted Or something like this ;) Stef
list Dave Haertig
p.s. - I posted these scripts a monent ago. I see in the email replay
that my cut-n-paste did not work 100%. It appears that everything is
there, but some newlines were lost in a few places. e.g., you will see:
${BBCMD} --env=${ENVFILE} ${CMD} >${LOGFILE} 2>&1 exit 0
... but if you know shell scripts, you'll recognize that the "exit 0"
part should be on a line by itself. I do not know why cut-n-paste
didn't work right. I guess some Linux/Solaris/Windows/Thunderbird
glitch. There are a few other places where commands were run-on like
the above example. Watch for them, and manually fix them.
Sorry about that!
▸
-----Original Message----- From: Haertig, David F (Dave) Sent: Monday, June 25, 2007 9:20 PM To: 'user-ae9b8668bcde@xymon.invalid' Subject: RE: [hobbit] hobbitfetch replacement with ssh Below is what I ended up doing. It's working like a champ. It probably doesn't fit your environment exactly, but it may be of some help anyway. My Hobbit server is Linux 2.6, the clients I'm pulling data from are Solaris 9. All running OpenSSH. The client domains are picked up on the Hobbit server via /etc/resolv.conf mechanism. Hobbit server has socks client installed and configured for our environment. ssh public keys were distributed from host to clients so ssh/scp will work without passwords being required. The file "hobbitclient-custom.sh" you see below is nothing more than the standard hobbitclient.sh file with some stuff I didn't want stripped out. I modified the LOGFETCHCFG line to point to the "scripts" directory rather than the default "tmp" directory because I wanted to keep all my customized stuff in that scripts directory. Custom HOST script, "pulldata.sh", found on server in $BBHOME/ext directory, run via clientlaunch.cfg ======================================================================== ============================= export SOCKS5_USER='my_socks_userid' export SOCKS5_PASSWD='my_socks_password' USER=stars for DMZ_MACH in 'machine1' 'machine2' 'machine3' do TMPFILE=${BBTMP}/pulldata.${DMZ_MACH}.tmp.$$ runsocks ssh ${USER}@${DMZ_MACH} hobbit/scripts/exec_client.sh runsocks scp ${USER}@${DMZ_MACH}:~${USER}/hobbit/tmp/msg.${DMZ_MACH}.txt ${TMPFILE} ${BB} ${BBDISP} "`cat ${TMPFILE}`" rm -f ${TMPFILE} done Custom CLIENT script, "exec_client.sh", found on client in /home/stars/hobbit/scripts directory ======================================================================== ======================= #!/bin/sh BBHOME=/home/stars/hobbit BBCMD=${BBHOME}/bin/bbcmd CMD=${BBHOME}/scripts/hobbitclient-custom.sh ENVFILE=${BBHOME}/etc/hobbitclient.cfg LOGFILE=${BBHOME}/logs/hobbitclient-custom.log ${BBCMD} --env=${ENVFILE} ${CMD} >${LOGFILE} 2>&1 exit 0 Custom CLIENT script, "hobbitclient-custom.sh", found on client in /home/stars/hobbit/scripts directory ======================================================================== =============================== #!/bin/sh LANG=C LC_ALL=C LC_MESSAGES=C export LANG LC_ALL LC_MESSAGES if test "$BBOSSCRIPT" = ""; then BBOSSCRIPT="hobbitclient-`uname -s | tr '[A-Z]' '[a-z]'`.sh" fi MSGFILE="$BBTMP/msg.$MACHINEDOTS.txt" MSGTMPFILE="$MSGFILE.$$" LOGFETCHCFG=$BBHOME/scripts/logfetch.$MACHINEDOTS.cfg LOGFETCHSTATUS=$BBTMP/logfetch.$MACHINEDOTS.status export LOGFETCHCFG LOGFETCHSTATUS rm -f $MSGTMPFILE touch $MSGTMPFILE echo "client $MACHINE.$BBOSTYPE $CONFIGCLASS" >>$MSGTMPFILE $BBHOME/bin/$BBOSSCRIPT >> $MSGTMPFILE if test -f $LOGFETCHCFG then $BBHOME/bin/logfetch $LOGFETCHCFG $LOGFETCHSTATUS >>$MSGTMPFILE fi echo "[clock]" >>$MSGTMPFILE $BBHOME/bin/logfetch --clock >>$MSGTMPFILE rm -f $MSGFILE mv $MSGTMPFILE $MSGFILE exit 0 Custom CLIENT file, "logfetch.machine1.txt", found on client in /home/stars/hobbit/scripts directory ======================================================================== ============================ log:/my_app/logs/file1:4096 log:/my_app/logs/file2:4096 log:/my_app/logs/file3:4096 -----Original Message----- From: John G [mailto:user-7da77b391823@xymon.invalid] Sent: Monday, June 25, 2007 8:30 PM To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] hobbitfetch replacement with ssh Here are some old messages on the topic that are similar to what you are doing. http://www.hswn.dk/hobbiton/2006/01/msg00045.html http://www.hswn.dk/hobbiton/2006/01/msg00047.html On 6/23/07, Stef Coene <user-dbffe946c0f4@xymon.invalid> wrote:
On Saturday 23 June 2007, Haertig, David F (Dave) wrote:Using bbcmd to run hobbitclient.sh was a good idea, but presented some problems of it's own. No problem getting it to set the environment though. For one, invoking the client in this manner leaves lots of iostatcpu, iostatdisk, and vmstat files laying around in the tmp directory. I guess it must be hobbitlaunch that cleans these up normally. Second, I could not trick logfetch into grabbing custom logfiles. I
tried putting the appropriate entries in ..../tmp/logfetch.XXX.cfg on the client end, but logfetch ignored these. This file is normally overwritten each time the client contacts the host, so normally you wouldn't edit it. But in my case the client is not connecting to the host so I thought it might be "safe" to manually create this file, but alas logfetch appears to ignore it. I may well run into this logfetch gotcha no matter how I invoke hobbitclient.sh If the client cannot talk to the Hobbit server, as in my DMZ case, is it even possible to configure logfetch to grab custom logfiles? The logfetch config is normally downloaded from the server. I don't see any way (yet) to configure it locally on the client.If "I" should have this problem, I will try this: change hobbit client.sh so it: - is not trying to contact the hobbit server (so the bb command is not used) - the message file is saved in a fixed file (so the pid is not appended to it) - the logfetch can still work if you update the input file manually make a daemon on the hobbit server: - that runs every minute - contacts the hobbit client with ssh and checks for a message file, if a message file is found, the content is send to the hobbit server and the file is deleted Or something like this ;) Stef
list Stef Coene
▸
On Tuesday 26 June 2007, Haertig, David F (Dave) wrote:
Below is what I ended up doing. It's working like a champ. It probably doesn't fit your environment exactly, but it may be of some help anyway. My Hobbit server is Linux 2.6, the clients I'm pulling data from are Solaris 9. All running OpenSSH. The client domains are picked up on the Hobbit server via /etc/resolv.conf mechanism. Hobbit server has socks client installed and configured for our environment. ssh public keys were distributed from host to clients so ssh/scp will work without passwords being required. The file "hobbitclient-custom.sh" you see below is nothing more than the standard hobbitclient.sh file with some stuff I didn't want stripped out. I modified the LOGFETCHCFG line to point to the "scripts" directory rather than the default "tmp" directory because I wanted to keep all my customized stuff in that scripts directory.
I was so free to upload the 4 files to my twiki, I hope you don't mine. This is a nice solution to a problem that more then 1 person will encounter: http://www.docum.org/twiki/bin/view/Hobbit/PullingSshClient Stef