ncv/splitncv question(s) 🔗 link

Hi,

After creating custom graphs with NCV for our san's storage processor load
I wanted to graph the read & write iops of our san on a per-lun basis. I
hoped it could be done kind of dynamic (like disk) with NCV but that seems
impossible. My san management station is spitting out nice status messages
(using bbwincmd) formatted like this:

<hobbit_display> status san lun,<lun#> green <timestamp> ReadIOPS :
<#oips> WriteIOPS : <#iops>

which result in numerous new columns under 'san' like "lun,11" "lun,12"
etc (used the ",<#>" format since the 'disk' rrd's seem to use it). I was
hoping they would all be 'hidden' in one 'lun' column (like disk) but that
was wishfull thinking. Any solution/fix for this is welcome. Clicking one
of these columns does show the correct results however (...although no
graph since it doesn't even make rrd's).

After reading about SPLITNCV in 
http://www.hswn.dk/hobbiton/2008/10/msg00423.html I could put something
like

SPLITNCV_lun="ReadIOPS:GAUGE,WriteIOPS:GAUGE"

in my hobbitserver.cfg but if I understand correctly this would result in
2 rrd files lun.ReadIOPS.rrd and lun.WriteIOPS.rrd and that's not what I'm
looking for. I'm hoping someone here "had fun" with something like this
before and is able to point me in the right direction since I'm at the
point that my head is about to explode.

Thanks in advance,

Wally

Wally.nl a écrit :

From - Tue
  

Hi,

What do you want, exactly ?
For information, SPLITNCV (and NCV too) needs a pre-formated output :
NAME1: VALUE1
NAME2: VALUE2
You need a new line character at this end of your output (don't ask me 
why...)
You can use NAME= VALUE instead of NAME: VALUE.

An advice, if your script sends data in the right format, you can use
SPLITNCV_test=*:GAUGE
it allows to get any value stored in RRD file.

In hobbitgraphs.cfg, you can use the following syntax :
[test]
        TITLE My I/O test
        YAXIS I/O per second
        FNPATTERN test,(.*).rrd
        DEF:@RRDIDX@=@RRDFN@:lambda:AVERAGE
        LINE2:@RRDIDX@#@COLOR@:@RRDPARAM@
        GPRINT:@RRDIDX@:LAST: \: %5.1lf (cur)
        GPRINT:@RRDIDX@:MAX: \: %5.1lf (max)
        GPRINT:@RRDIDX@:MIN: \: %5.1lf (min)
        GPRINT:@RRDIDX@:AVERAGE: \: %5.1lf (avg)\n


I hope it could help you.
I think it could be usefull to know what you exactly want for final 
result. (multiple graphs, one graph with multiple lines...)

What do you want, exactly ?

Something impossible probably :)

I want multiple graphs with 2 lines each and I get the feeling that
SPLITNCV is limited to 1 line / graph.

I hope the attached screenshots clarify things. If I send status for just
1 lun there is no problem (attached SingleGraph.jpg), I get a column, RRD
file and working graph. If I send the data for multiple luns I get
multiple columns (attached columns.jpg) instead of a single one (even if I
send it in "lun,<#>" format like 'disk' seems to do). But a bigger issue
is that I don't get any RRD files/graphs if I do that. It appears that the
NCV logic is limited to 1 RRD/graph somehow where I was hoping it would be
as flexible as the disk monitor.

What I would like as an end-result is a single 'lun' column which
'unfolds' like disk into multiple graphs something like the attached
'MultipleMockup.jpg' although I thing that even the graph title is an
issue since it doesn't like parse stuff like @RRDFN@ in the title field.

I _really_ hope someone has some form of solution/workaround/patch/ideas.

Wally.nl

Wally.nl a écrit :

Hi,

As far as i know, you can have only one graph illustrating a test.
 But you can define in your hobbitgraph.cfg several graphs using your 
rrd files. So you could define like this :

[lun]
FNPATTERN lun(.*).rrd
TITLE LUNs IOPS
YAXIS IOPS
DEF:p at RRDIDX@=@RRDFN@:lambda:AVERAGE
LINE2:p at RRDIDX@#@COLOR@:@RRDPARAM@
GPRINT:p at RRDIDX@:LAST: \: %5.1lf (cur)
GPRINT:p at RRDIDX@:MAX: \: %5.1lf (max)
GPRINT:p at RRDIDX@:MIN: \: %5.1lf (min)
GPRINT:p at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n

[lun1]
FNPATTERN lun1(.*).rrd
TITLE LUN1 IOPS
YAXIS IOPS
...

then you could define in your hobbitserver.cfg :
GRAPHS="...,lun1,lun2,..."

and finally in bb-hosts :
host IP.AD.DR.ESS # TRENDS:*,lun1,lun2...

So, you'll have a graph dedicated for "lun" column, including all IOPS 
(maybe you could format your test's output to get comprehensive name as 
"Read_lun1").
And in "trend" column, you'll get separated graphs
If I misunderstood your request, please add following details :
-your script output format
-your actual [lunX] definitions in hobbitgraph.cfg, and SPLITNCV 
definitions for lun* in hobbitserver.cfg

Hope this helps

Thanks for your suggestions, I had already tried parts of them but so far
none of them make me happy.

I'm currently looking at the data/trend method described here:
http://www.hswn.dk/hobbiton/2007/06/msg00250.html

It seems like an easy way to create the different RRDs, just need to
figure out how to graph them but that's probably not too difficult.

I'll first have to fix bbwincmd (data is send from a windoze box) because
the stock version/source is useless (can't send the needed \n in the
status/data commands). I'll get back if there's some progress to report.

Wally.nl

Wally.nl a écrit :

In my limited knowledges about hobbit and NCV module (i don't have any C 
or system low level skills), you can get multiple files in formating 
your test output like this :

your text
anything you want
....
lun0_input : value1
lun0_output : value2
lun1_input : value3
lun1_output : value4
NEW LINE CHARACTER


So you can define your hobbitserver.cvf as it :
SPLITNCV_lun="*:GAUGE" (or any other RRD data type)
GRAPHS="...,lun0,lun1,lun2..."

You'll have several RRD files, named like this :
lun,lun0_input.rrd
lun,lun0_ouput.rrd
lun,lun1_input.rrd
...
then you can define hobbitgraph.cfg as I did :
[lun]
         FNPATTERN lun,(.*).rrd
         TITLE LUNs IOS
         YAXIS IOS

         DEF:p at RRDIDX@=@RRDFN@:lambda:AVERAGE
         LINE2:p at RRDIDX@#@COLOR@:@RRDPARAM@
         GPRINT:p at RRDIDX@:LAST: \: %5.1lf (cur)
         GPRINT:p at RRDIDX@:MAX: \: %5.1lf (max)
         GPRINT:p at RRDIDX@:MIN: \: %5.1lf (min)
         GPRINT:p at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n

You can define per lun graphs :

[lun0]
	FNPATTERN lun,lun0(.*).rrd
         TITLE LUN0 IOS
         YAXIS IOS

         DEF:p at RRDIDX@=@RRDFN@:lambda:AVERAGE
         LINE2:p at RRDIDX@#@COLOR@:@RRDPARAM@
         GPRINT:p at RRDIDX@:LAST: \: %5.1lf (cur)
         GPRINT:p at RRDIDX@:MAX: \: %5.1lf (max)
         GPRINT:p at RRDIDX@:MIN: \: %5.1lf (min)
         GPRINT:p at RRDIDX@:AVERAGE: \: %5.1lf (avg)\n

in bb-hosts, define :
IP.AD.DR.ESS hostname # TRENDS:*,lun0,lun1...

So, you'll have a global graph including all LUNs IOS, drawed when you 
click lun test.
And you'll have several lun0, lun1 ... graphs drawed in trends.

If that's not the result you'd like, please be more specific (and don't 
forget I'm french, so I suck in foreign langages)