Xymon Mailing List Archive search

/snap/core/* Filesystems NOT ok

5 messages in this thread

list George Michalopoulos · Mon, 4 Sep 2023 15:25:09 +0300 ·
hello all,
i have the known problem with /snap/*
Mon Sep 4 14:33:50 EEST 2023 - Filesystems NOT ok

[image: red] /snap/core/15511 (100% used) has reached the PANIC level
(95%)[image: red] /snap/core20/1974 (100% used) has reached the PANIC
level (95%)[image: red] /snap/certbot/3024 (100% used) has reached the
PANIC level (95%)[image: red] /snap/certbot/2913 (100% used) has
reached the PANIC level (95%)[image: red] /snap/core20/2015 (100%
used) has reached the PANIC level (95%)[image: red] /snap/core/15925
(100% used) has reached the PANIC level (95%)

Filesystem     1024-blocks     Used Available Capacity Mounted on
/dev/sda1         98497780 25367880  68080352      28% /
tmpfs                 5120        4      5116       1% /run/lock
/dev/loop1          121088   121088         0     100% /snap/core/15511
/dev/loop0           65024    65024         0     100% /snap/core20/1974
/dev/loop2           45440    45440         0     100% /snap/certbot/3024
/dev/loop3           45696    45696         0     100% /snap/certbot/2913
/dev/loop6           65024    65024         0     100% /snap/core20/2015
/dev/loop4          108416   108416         0     100% /snap/core/15925

I added this to my analysis.cfg no good...

HOST=*
    DISK %^/snap IGNORE
    INODE %^/snap/.+ IGNORE

this is my default analysis.cfg


CLASS=linux
        # Ignore some usually uninteresting tmpfs mounts.
        DISK    /dev IGNORE
        DISK    /snap IGNORE
        DISK    /lib/init/rw IGNORE
        DISK    /sys/fs/cgroup IGNORE
        DISK    %^/run/user/\d+$ IGNORE
        INODE   /dev IGNORE
        INODE   /snap IGNORE
        INODE   /lib/init/rw IGNORE
        INODE   /sys/fs/cgroup IGNORE
        INODE   %^/run/user/\d+$ IGNORE


DEFAULT
        # Ignore some usually uninteresting tmpfs mounts.
        DISK    /dev IGNORE
        DISK    /dev/shm IGNORE
        DISK    /lib/init/rw IGNORE
        DISK    /run IGNORE
        # These are the built-in defaults. You should only modify these
        # lines, not add new ones (no PROC, DISK, LOG ... lines).
        UP      1h
        LOAD    5.0 10.0
        DISK    * 90 95
        INODE   * 70 90
        MEMPHYS 100 101
        MEMSWAP 50 80
        MEMACT  90 97


-- 

regards,
George Michalopoulos

born & raise @ Metron 6 (six)

user-17e4b93d552a@xymon.invalid
list Torsten Richter · Mon, 4 Sep 2023 14:44:31 +0200 ·
Hi,

have you tried without the regex?
I have here

HOST=xyz
     DISK /mnt IGNORE
     INODE /mnt IGNORE

which is working.

Regards,
Torsten
quoted from George Michalopoulos

On 04.09.2023 14:25, George Michalopoulos wrote:
hello all,
i have the known problem with /snap/*
Mon Sep 4 14:33:50 EEST 2023 - Filesystems NOT ok

[image: red] /snap/core/15511 (100% used) has reached the PANIC level
(95%)[image: red] /snap/core20/1974 (100% used) has reached the PANIC
level (95%)[image: red] /snap/certbot/3024 (100% used) has reached the
PANIC level (95%)[image: red] /snap/certbot/2913 (100% used) has
reached the PANIC level (95%)[image: red] /snap/core20/2015 (100%
used) has reached the PANIC level (95%)[image: red] /snap/core/15925
(100% used) has reached the PANIC level (95%)

Filesystem     1024-blocks     Used Available Capacity Mounted on
/dev/sda1         98497780 25367880  68080352      28% /
tmpfs                 5120        4      5116       1% /run/lock
/dev/loop1          121088   121088         0     100% /snap/core/15511
/dev/loop0           65024    65024         0     100% /snap/core20/1974
/dev/loop2           45440    45440         0     100% /snap/certbot/3024
/dev/loop3           45696    45696         0     100% /snap/certbot/2913
/dev/loop6           65024    65024         0     100% /snap/core20/2015
/dev/loop4          108416   108416         0     100% /snap/core/15925

I added this to my analysis.cfg no good...

HOST=*
     DISK %^/snap IGNORE
     INODE %^/snap/.+ IGNORE

this is my default analysis.cfg


CLASS=linux
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /snap IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /sys/fs/cgroup IGNORE
         DISK    %^/run/user/\d+$ IGNORE
         INODE   /dev IGNORE
         INODE   /snap IGNORE
         INODE   /lib/init/rw IGNORE
         INODE   /sys/fs/cgroup IGNORE
         INODE   %^/run/user/\d+$ IGNORE


DEFAULT
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /dev/shm IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /run IGNORE
         # These are the built-in defaults. You should only modify these
         # lines, not add new ones (no PROC, DISK, LOG ... lines).
         UP      1h
         LOAD    5.0 10.0
         DISK    * 90 95
         INODE   * 70 90
         MEMPHYS 100 101
         MEMSWAP 50 80
         MEMACT  90 97

-- 
+---------------------------------------------------------+

| E-mail  : user-c862b499d9fa@xymon.invalid			  |
|							  |
| Homepage: https://www.unixadmin.de/			  |
+---------------------------------------------------------+
Download my public key from:
http://keys.gnupg.net/pks/lookup?search=0x899093AC&op=get
list Becker Christian · Mon, 4 Sep 2023 13:00:24 +0000 ·
Hello there,

in our setup, for every single host that has these snap-things, we have:

HOST=hostname
            DISK    %^/snap/.+ IGNORE
            INODE   %^/snap/.+ IGNORE

And nothing with a wildcard.
Think i can remember that if you have an entry with a wildcard, and after that another entry with an explicit hostname, that last found explicit entry COULD override the one with the wildcard; but not really sure about that, but worth a try.

Regards
Christian


-----Urspr?ngliche Nachricht-----
Von: Xymon <xymon-bounces at xymon.com> Im Auftrag von Torsten Richter
Gesendet: Montag, 4. September 2023 14:45
An: George Michalopoulos <user-17e4b93d552a@xymon.invalid>; xymon at xymon.com
Betreff: Re: [Xymon] /snap/core/* Filesystems NOT ok

WARNUNG: Diese E-Mail kam von au?erhalb der Organisation. Klicken Sie nicht auf Links oder ?ffnen Sie keine Anh?nge, es sei denn, Sie kennen den Absender und wissen, dass der Inhalt sicher ist.
quoted from Torsten Richter


Hi,

have you tried without the regex?
I have here

HOST=xyz
     DISK /mnt IGNORE
     INODE /mnt IGNORE

which is working.

Regards,
Torsten

On 04.09.2023 14:25, George Michalopoulos wrote:
hello all,
i have the known problem with /snap/*
Mon Sep 4 14:33:50 EEST 2023 - Filesystems NOT ok

[image: red] /snap/core/15511 (100% used) has reached the PANIC level
(95%)[image: red] /snap/core20/1974 (100% used) has reached the PANIC
level (95%)[image: red] /snap/certbot/3024 (100% used) has reached the
PANIC level (95%)[image: red] /snap/certbot/2913 (100% used) has
reached the PANIC level (95%)[image: red] /snap/core20/2015 (100%
used) has reached the PANIC level (95%)[image: red] /snap/core/15925
(100% used) has reached the PANIC level (95%)

Filesystem     1024-blocks     Used Available Capacity Mounted on
/dev/sda1         98497780 25367880  68080352      28% /
tmpfs                 5120        4      5116       1% /run/lock
/dev/loop1          121088   121088         0     100% /snap/core/15511
/dev/loop0           65024    65024         0     100% /snap/core20/1974
/dev/loop2           45440    45440         0     100% /snap/certbot/3024
/dev/loop3           45696    45696         0     100% /snap/certbot/2913
/dev/loop6           65024    65024         0     100% /snap/core20/2015
/dev/loop4          108416   108416         0     100% /snap/core/15925

I added this to my analysis.cfg no good...

HOST=*
     DISK %^/snap IGNORE
     INODE %^/snap/.+ IGNORE

this is my default analysis.cfg


CLASS=linux
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /snap IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /sys/fs/cgroup IGNORE
         DISK    %^/run/user/\d+$ IGNORE
         INODE   /dev IGNORE
         INODE   /snap IGNORE
         INODE   /lib/init/rw IGNORE
         INODE   /sys/fs/cgroup IGNORE
         INODE   %^/run/user/\d+$ IGNORE


DEFAULT
         # Ignore some usually uninteresting tmpfs mounts.
         DISK    /dev IGNORE
         DISK    /dev/shm IGNORE
         DISK    /lib/init/rw IGNORE
         DISK    /run IGNORE
         # These are the built-in defaults. You should only modify these
         # lines, not add new ones (no PROC, DISK, LOG ... lines).
         UP      1h
         LOAD    5.0 10.0
         DISK    * 90 95
         INODE   * 70 90
         MEMPHYS 100 101
         MEMSWAP 50 80
         MEMACT  90 97


http://lists/
.xymon.com%2Fmailman%2Flistinfo%2Fxymon&data=05%7C01%7Cchristian.becke
user-539421219919@xymon.invalid%7Ca7b60ebd3adc41cb757108dbad4567f3%7C4fed923898b
c4f3b96450b99f4d1b669%7C0%7C0%7C638294285832667196%7CUnknown%7CTWFpbGZ
sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
D%7C3000%7C%7C%7C&sdata=9QAhckVq9i7t671SX2s%2B%2BqKkNUoWmEv4NiQSkORwzm
c%3D&reserved=0
--
+---------------------------------------------------------+
| E-mail  : user-c862b499d9fa@xymon.invalid                       |
|                                                         |
| Homepage: https://www.unixadmin.de/                     |
+---------------------------------------------------------+
Download my public key from:
http://keys.gnupg.net/pks/lookup?search=0x899093AC&op=get
list Adam Thorn · Mon, 4 Sep 2023 17:09:00 +0100 ·
quoted from Becker Christian
On 04/09/2023 14:00, Becker Christian wrote:
Hello there,

in our setup, for every single host that has these snap-things, we have:

HOST=hostname
             DISK    %^/snap/.+ IGNORE
             INODE   %^/snap/.+ IGNORE

And nothing with a wildcard.
Think i can remember that if you have an entry with a wildcard, and after that another entry with an explicit hostname, that last found explicit entry COULD override the one with the wildcard; but not really sure about that, but worth a try.
See "man analysis.cfg":

"The  entire  file  is evaluated from the top to bottom, and the first match found is used. So you should put the specific settings first, and the generic ones last."

A line for "DISK *" thus risks masking later rules for any "DISK /name/of/mountpoint", subject to how the HOST/PAGE/etc matching rules are defined.

You can test this with an example analysis.cfg file by running

xymond_client --config=/tmp/analysis.cfg --test

which will give you an interactive prompt to see how your analysis.cfg is working.  (NB it seems that the argument order matters; if you want to specify an experimental config file rather than using the default then the --config option has to come first, at least on a Ubuntu 20.04 install!)

If I use:

== begin bad analysis.cfg ==
DEFAULT
   DISK * 90 95

HOST=*
   DISK %^/snap/.+ IGNORE
== end bad analysis.cfg ==

then the default thresholds of 90/95% are applied to /snap/test. But swapping the order:

== begin good analysis.cfg ==
HOST=*
   DISK %^/snap/.+ IGNORE

DEFAULT
   DISK * 90 95
== end good analysis.cfg ==

means the first ("IGNORE") rule gets matched first for /snap/test as desired.

Adam
list Lars Kollstedt · Mon, 11 Sep 2023 10:08:44 +0200 ·
Hi George,

On 04.09.23 14:25, George Michalopoulos wrote:
HOST=*
 ? ? DISK %^/snap IGNORE
 ? ? INODE %^/snap/.+ IGNORE
I'm not sure if a wildcard, without regex is supported in HOST.

Possibly this will have to be

HOST=%.*
 ? ? DISK %^/snap/ IGNORE
 ??? INODE %^/snap/ IGNORE

the `DISK ? ?/snap IGNORE` in CLASS=linux will probably only match /snap 
exactly not the mountpoints in the subdirectories like? /snap/core/15511.

Kind regards

 ??? Lars

-- 
Lars Kollstedt

Telefon: +49 6151 16-71027
E-Mail:  user-0f90394071da@xymon.invalid

man-da.de GmbH
Dolivostra?e 11
64293 Darmstadt

Sitz der Gesellschaft: Darmstadt
Registergericht: Amtsgericht Darmstadt
Handelsregisternummer: HRB 9484
Gesch?ftsf?hrer: Andreas Ebert