Xymon Mailing List Archive search

/snap/core/* Filesystems NOT ok

5 messages in this thread

list George Michalopoulos · Mon, 4 Sep 2023 15:25:09 +0300 ·
hello all,
i have the known problem with /snap/*
Mon Sep 4 14:33:50 EEST 2023 - Filesystems NOT ok

[image: red] /snap/core/15511 (100% used) has reached the PANIC level
(95%)[image: red] /snap/core20/1974 (100% used) has reached the PANIC
level (95%)[image: red] /snap/certbot/3024 (100% used) has reached the
PANIC level (95%)[image: red] /snap/certbot/2913 (100% used) has
reached the PANIC level (95%)[image: red] /snap/core20/2015 (100%
used) has reached the PANIC level (95%)[image: red] /snap/core/15925
(100% used) has reached the PANIC level (95%)

Filesystem     1024-blocks     Used Available Capacity Mounted on
/dev/sda1         98497780 25367880  68080352      28% /
tmpfs                 5120        4      5116       1% /run/lock
/dev/loop1          121088   121088         0     100% /snap/core/15511
/dev/loop0           65024    65024         0     100% /snap/core20/1974
/dev/loop2           45440    45440         0     100% /snap/certbot/3024
/dev/loop3           45696    45696         0     100% /snap/certbot/2913
/dev/loop6           65024    65024         0     100% /snap/core20/2015
/dev/loop4          108416   108416         0     100% /snap/core/15925

I added this to my analysis.cfg no good...

HOST=*
    DISK %^/snap IGNORE
    INODE %^/snap/.+ IGNORE

this is my default analysis.cfg


CLASS=linux
        # Ignore some usually uninteresting tmpfs mounts.
        DISK    /dev IGNORE
        DISK    /snap IGNORE
        DISK    /lib/init/rw IGNORE
        DISK    /sys/fs/cgroup IGNORE
        DISK    %^/run/user/\d+$ IGNORE
        INODE   /dev IGNORE
        INODE   /snap IGNORE
        INODE   /lib/init/rw IGNORE
        INODE   /sys/fs/cgroup IGNORE
        INODE   %^/run/user/\d+$ IGNORE


DEFAULT
        # Ignore some usually uninteresting tmpfs mounts.
        DISK    /dev IGNORE
        DISK    /dev/shm IGNORE
        DISK    /lib/init/rw IGNORE
        DISK    /run IGNORE
        # These are the built-in defaults. You should only modify these
        # lines, not add new ones (no PROC, DISK, LOG ... lines).
        UP      1h
        LOAD    5.0 10.0
        DISK    * 90 95
        INODE   * 70 90
        MEMPHYS 100 101
        MEMSWAP 50 80
        MEMACT  90 97


-- 

regards,
George Michalopoulos

born & raise @ Metron 6 (six)

user-17e4b93d552a@xymon.invalid
list Torsten Richter · Mon, 4 Sep 2023 14:44:31 +0200 ·
Hi,

have you tried without the regex?
I have here

HOST=xyz
     DISK /mnt IGNORE
     INODE /mnt IGNORE

which is working.

Regards,
Torsten
▸ quoted from George Michalopoulos

On 04.09.2023 14:25, George Michalopoulos wrote:
hello all,
i have the known problem with /snap/*
Mon Sep 4 14:33:50 EEST 2023 - Filesystems NOT ok

[image: red] /snap/core/15511 (100% used) has reached the PANIC level
(95%)[image: red] /snap/core20/1974 (100% used) has reached the PANIC
level (95%)[image: red] /snap/certbot/3024 (100% used) has reached the
PANIC level (95%)[image: red] /snap/certbot/2913 (100% used) has
reached the PANIC level (95%)[image: red] /snap/core20/2015 (100%
used) has reached the PANIC level (95%)[image: red] /snap/core/15925
(100% used) has reached the PANIC level (95%)

Filesystem     1024-blocks     Used Available Capacity Mounted on
/dev/sda1         98497780 25367880  68080352      28% /
tmpfs                 5120        4      5116       1% /run/lock
/dev/loop1          121088   121088         0     100% /snap/core/15511
/dev/loop0           65024    65024         0     100% /snap/core20/1974
/dev/loop2           45440    45440         0     100% /snap/certbot/3024
/dev/loop3           45696    45696         0     100% /snap/certbot/2913
/dev/loop6           65024    65024         0     100% /snap/core20/2015
/dev/loop4          108416   108416         0     100% /snap/core/15925

I added this to my analysis.cfg no good...

HOST=*
     DISK %^/snap IGNORE
     INODE %^/snap/.+ IGNORE

this is my default analysis.cfg


CLASS=linux
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /snap IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /sys/fs/cgroup IGNORE
         DISK    %^/run/user/\d+$ IGNORE
         INODE   /dev IGNORE
         INODE   /snap IGNORE
         INODE   /lib/init/rw IGNORE
         INODE   /sys/fs/cgroup IGNORE
         INODE   %^/run/user/\d+$ IGNORE


DEFAULT
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /dev/shm IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /run IGNORE
         # These are the built-in defaults. You should only modify these
         # lines, not add new ones (no PROC, DISK, LOG ... lines).
         UP      1h
         LOAD    5.0 10.0
         DISK    * 90 95
         INODE   * 70 90
         MEMPHYS 100 101
         MEMSWAP 50 80
         MEMACT  90 97

-- 
+---------------------------------------------------------+

| E-mail  : user-c862b499d9fa@xymon.invalid			  |
|							  |
| Homepage: https://www.unixadmin.de/			  |
+---------------------------------------------------------+
Download my public key from:
http://keys.gnupg.net/pks/lookup?search=0x899093AC&op=get
list Becker Christian · Mon, 4 Sep 2023 13:00:24 +0000 ·
Hello there,

in our setup, for every single host that has these snap-things, we have:

HOST=hostname
            DISK    %^/snap/.+ IGNORE
            INODE   %^/snap/.+ IGNORE

And nothing with a wildcard.
Think i can remember that if you have an entry with a wildcard, and after that another entry with an explicit hostname, that last found explicit entry COULD override the one with the wildcard; but not really sure about that, but worth a try.

Regards
Christian


-----Urspr?ngliche Nachricht-----
Von: Xymon <xymon-bounces at xymon.com> Im Auftrag von Torsten Richter
Gesendet: Montag, 4. September 2023 14:45
An: George Michalopoulos <user-17e4b93d552a@xymon.invalid>; xymon at xymon.com
Betreff: Re: [Xymon] /snap/core/* Filesystems NOT ok

WARNUNG: Diese E-Mail kam von au?erhalb der Organisation. Klicken Sie nicht auf Links oder ?ffnen Sie keine Anh?nge, es sei denn, Sie kennen den Absender und wissen, dass der Inhalt sicher ist.
▸ quoted from Torsten Richter


Hi,

have you tried without the regex?
I have here

HOST=xyz
     DISK /mnt IGNORE
     INODE /mnt IGNORE

which is working.

Regards,
Torsten

On 04.09.2023 14:25, George Michalopoulos wrote:
hello all,
i have the known problem with /snap/*
Mon Sep 4 14:33:50 EEST 2023 - Filesystems NOT ok

[image: red] /snap/core/15511 (100% used) has reached the PANIC level
(95%)[image: red] /snap/core20/1974 (100% used) has reached the PANIC
level (95%)[image: red] /snap/certbot/3024 (100% used) has reached the
PANIC level (95%)[image: red] /snap/certbot/2913 (100% used) has
reached the PANIC level (95%)[image: red] /snap/core20/2015 (100%
used) has reached the PANIC level (95%)[image: red] /snap/core/15925
(100% used) has reached the PANIC level (95%)

Filesystem     1024-blocks     Used Available Capacity Mounted on
/dev/sda1         98497780 25367880  68080352      28% /
tmpfs                 5120        4      5116       1% /run/lock
/dev/loop1          121088   121088         0     100% /snap/core/15511
/dev/loop0           65024    65024         0     100% /snap/core20/1974
/dev/loop2           45440    45440         0     100% /snap/certbot/3024
/dev/loop3           45696    45696         0     100% /snap/certbot/2913
/dev/loop6           65024    65024         0     100% /snap/core20/2015
/dev/loop4          108416   108416         0     100% /snap/core/15925

I added this to my analysis.cfg no good...

HOST=*
     DISK %^/snap IGNORE
     INODE %^/snap/.+ IGNORE

this is my default analysis.cfg


CLASS=linux
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /snap IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /sys/fs/cgroup IGNORE
         DISK    %^/run/user/\d+$ IGNORE
         INODE   /dev IGNORE
         INODE   /snap IGNORE
         INODE   /lib/init/rw IGNORE
         INODE   /sys/fs/cgroup IGNORE
         INODE   %^/run/user/\d+$ IGNORE


DEFAULT
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /dev/shm IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /run IGNORE
         # These are the built-in defaults. You should only modify these
         # lines, not add new ones (no PROC, DISK, LOG ... lines).
         UP      1h
         LOAD    5.0 10.0
         DISK    * 90 95
         INODE   * 70 90
         MEMPHYS 100 101
         MEMSWAP 50 80
         MEMACT  90 97


http://lists/
.xymon.com%2Fmailman%2Flistinfo%2Fxymon&data=05%7C01%7Cchristian.becke
user-539421219919@xymon.invalid%7Ca7b60ebd3adc41cb757108dbad4567f3%7C4fed923898b
c4f3b96450b99f4d1b669%7C0%7C0%7C638294285832667196%7CUnknown%7CTWFpbGZ
sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
D%7C3000%7C%7C%7C&sdata=9QAhckVq9i7t671SX2s%2B%2BqKkNUoWmEv4NiQSkORwzm
c%3D&reserved=0
--
+---------------------------------------------------------+
| E-mail  : user-c862b499d9fa@xymon.invalid                       |
|                                                         |
| Homepage: https://www.unixadmin.de/                     |
+---------------------------------------------------------+
Download my public key from:
http://keys.gnupg.net/pks/lookup?search=0x899093AC&op=get
list Adam Thorn · Mon, 4 Sep 2023 17:09:00 +0100 ·
▸ quoted from Becker Christian
On 04/09/2023 14:00, Becker Christian wrote:
Hello there,

in our setup, for every single host that has these snap-things, we have:

HOST=hostname
             DISK    %^/snap/.+ IGNORE
             INODE   %^/snap/.+ IGNORE

And nothing with a wildcard.
Think i can remember that if you have an entry with a wildcard, and after that another entry with an explicit hostname, that last found explicit entry COULD override the one with the wildcard; but not really sure about that, but worth a try.
See "man analysis.cfg":

"The  entire  file  is evaluated from the top to bottom, and the first match found is used. So you should put the specific settings first, and the generic ones last."

A line for "DISK *" thus risks masking later rules for any "DISK /name/of/mountpoint", subject to how the HOST/PAGE/etc matching rules are defined.

You can test this with an example analysis.cfg file by running

xymond_client --config=/tmp/analysis.cfg --test

which will give you an interactive prompt to see how your analysis.cfg is working.  (NB it seems that the argument order matters; if you want to specify an experimental config file rather than using the default then the --config option has to come first, at least on a Ubuntu 20.04 install!)

If I use:

== begin bad analysis.cfg ==
DEFAULT
   DISK * 90 95

HOST=*
   DISK %^/snap/.+ IGNORE
== end bad analysis.cfg ==

then the default thresholds of 90/95% are applied to /snap/test. But swapping the order:

== begin good analysis.cfg ==
HOST=*
   DISK %^/snap/.+ IGNORE

DEFAULT
   DISK * 90 95
== end good analysis.cfg ==

means the first ("IGNORE") rule gets matched first for /snap/test as desired.

Adam
list Lars Kollstedt · Mon, 11 Sep 2023 10:08:44 +0200 ·
Hi George,

On 04.09.23 14:25, George Michalopoulos wrote:
HOST=*
 ? ? DISK %^/snap IGNORE
 ? ? INODE %^/snap/.+ IGNORE
I'm not sure if a wildcard, without regex is supported in HOST.

Possibly this will have to be

HOST=%.*
 ? ? DISK %^/snap/ IGNORE
 ??? INODE %^/snap/ IGNORE

the `DISK ? ?/snap IGNORE` in CLASS=linux will probably only match /snap 
exactly not the mountpoints in the subdirectories like? /snap/core/15511.

Kind regards

 ??? Lars

-- 
Lars Kollstedt

Telefon: +49 6151 16-71027
E-Mail:  user-0f90394071da@xymon.invalid

man-da.de GmbH
Dolivostra?e 11
64293 Darmstadt

Sitz der Gesellschaft: Darmstadt
Registergericht: Amtsgericht Darmstadt
Handelsregisternummer: HRB 9484
Gesch?ftsf?hrer: Andreas Ebert