Cleaning up history logs
list Michael Beatty
What does Xymon check to determine if my history logs directory is full? After several months of not paying any attention to Xymon, I dove back in. I found that I was getting "Historical Status Log Not Avaialable". Some browsing around showed that if the log directory has less than 5% disk space available, the logs will not be written. I "df -i /home/xymon/data/histlog" directory and it was (and still) is showing 4% used. Unsure what to get rid of, I deleted everything I could find in the data directory that related to old hosts that do not exist. After doing so, log files were now available. Obviously, there was a disk space problem, but where was it? This is just a demo system so I wasn't too worried about doing such a destructive deletion of everything, however, if this were a production system I'd want to be a lot more careful as to what it is that I delete. Would I just delete everything in the histlog directory that is > X days old? -- Michael Beatty Sherwin-Williams IT Analyst/Developer user-4aea7c115850@xymon.invalid XXX-XXX-XXXX
list Ryan Novosielski
See the "trimhistory" command. And there's nothing that says Xymon can't write after 5%, just the OS may have a certain amount of reserve space that is tunable, depending on the FS type and OS. Bear in mind Xymon must be restarted after a trimhistory command otherwise the "X changes in the last 240 minutes" list will remain blank. Also make sure you run the command as the Xymon user, not root.
▸
----- Original Message -----
From: Michael Beatty [mailto:user-4aea7c115850@xymon.invalid]
Sent: Tuesday, December 04, 2012 12:11 PM
To: xymon at xymon.com <xymon at xymon.com>
Subject: [Xymon] Cleaning up history logs
What does Xymon check to determine if my history logs directory is full?
After several months of not paying any attention to Xymon, I dove back in. I found that I was getting "Historical Status Log Not Avaialable". Some browsing around showed that if the log directory has less than 5% disk space available, the logs will not be written. I "df -i /home/xymon/data/histlog" directory and it was (and still) is showing 4% used. Unsure what to get rid of, I deleted everything I could find in the data directory that related to old hosts that do not exist. After doing so, log files were now available.
Obviously, there was a disk space problem, but where was it? This is just a demo system so I wasn't too worried about doing such a destructive deletion of everything, however, if this were a production system I'd want to be a lot more careful as to what it is that I delete. Would I just delete everything in the histlog directory that is > X days old?
--
Michael Beatty
Sherwin-Williams
IT Analyst/Developer
user-4aea7c115850@xymon.invalid
XXX-XXX-XXXX
list Henrik Størner
▸
On 04-12-2012 18:11, Michael Beatty wrote:
What does Xymon check to determine if my history logs directory is full?
It uses the "statvfs()" function, which should return how much space is available for unprivileged users (i.e. non-root users). Same info as a "df" will give you. If less than 5% is free, it will stop saving history logs. The threshold can be changed via the --minimum-free=N option to xymond_history.
▸
Obviously, there was a disk space problem, but where was it? This is just a demo system so I wasn't too worried about doing such a destructive deletion of everything, however, if this were a production system I'd want to be a lot more careful as to what it is that I delete. Would I just delete everything in the histlog directory that isX days old?
It is safe to remove files in the histlogs and hostdata directories. Using "trimhistory" is the recommended way of doing it. Regards, Henrik
list Ryan Novosielski
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
▸
On 12/04/2012 04:03 PM, Henrik Størner wrote:On 04-12-2012 18:11, Michael Beatty wrote:What does Xymon check to determine if my history logs directory is full?It uses the "statvfs()" function, which should return how much space is available for unprivileged users (i.e. non-root users). Same info as a "df" will give you. If less than 5% is free, it will stop saving history logs. The threshold can be changed via the --minimum-free=N option to xymond_history.
Interesting, so I guess I stand corrected! I guess it is possible this is happening to me on my Solaris install and I don't realize it? My drive is 99% full according to df, but I never understood this reserved space to be part of the "100%" of the drive -- after the drive is 100% free, there is still 5% left in blocks/kilobytes. - -- - ---- _ _ _ _ ___ _ _ _ |Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Sr. Systems Programmer |$&| |__| | | |__/ | \| _| |user-ae4522577e16@xymon.invalid - 973/972.0922 (2-0922) \__/ Univ. of Med. and Dent.|IST/EI-Academic Svcs. - ADMC 450, Newark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlC+ZzgACgkQmb+gadEcsb7UvgCfSGvbZHdK4dh+1itIScEIzML+ q5MAoLfSMJ0QfYblG1R9ySPKZNUzQLI+ =aRQt -----END PGP SIGNATURE-----