Splunk Dev

Can i divide a spark-line output into different actions ALIVE, Starting or Dead ?

robertlynch2020
Motivator

I am monitoring a system and i am getting data from it. In a 1 minute real time window, I add a sparkline and i can see a pulse.

SO below is = ALIVE.
alt text

The questions is how can i get Starting and DEAD.
If left to right is flat, but it has a pulse at the end = Starting

If right to left is flat and a pulse at the end = DEAD

Math can't help here as the Starting and Dead have the same numbers count. What I need is a way to catch the math in a vector.

From there, we run math on the vector and distinguish the difference from START - I think !

Any help would be great 🙂

I have this dont but i am using an alert to push data to a lookup table, but the solution is complex and i was hoping to remove the one minutes alerts and just use the data in real time.

alt text
Code i have to get image below.

index=jmx sourcetype=jmx host="hp548srv.fr.murex.com:9080" jvmDescription="*" mbean_domain="murex" 
| search source = *\=service* 
| rex field=source "murex:servicecode=(?<servicecode>.*),nickname=(?<nickname>.*),sid=(?<sid>.*),s" 
| eval source = servicecode." # ".nickname." # ".sid 
| chart sparkline count by source 
| rename source as Servicecode"#"Nickname"#"SID 
| rename sparkline as Heartbeat 
| rename count as status 
| lookup update=true Technical_View_Status_Services.csv Servicecode#Nickname#SID AS "Servicecode#Nickname#SID" OUTPUT status as status_history 
| eval epoch=mvindex(epoch,-1) 
| eval status_history=mvindex(status_history,-1) 
| eval status=if(status > 1, if(status > 2, "ALIVE", if(status_history="ALIVE","FLUX","STARTING")), if(isnull(status_history),"STARTING","DEAD")) 
| fields - status_history 
| search Servicecode#Nickname#SID = *** 
| appendpipe 
    [ stats count 
    | eval "NoResults"="No Services Found" 
    | where count=0 
    | table "NoResults"]

Robbie

1 Solution

woodcock
Esteemed Legend

This is NOT a complete answer but it should give you enough to work with to craft your own. Here is a run-anywhere search that generates an "ALIVE" sparklkine (set TimePicker to All time😞

| gentimes start=1/1/17 end=10/1/18 increment=1d
| rename starttime AS _time
| stats sparkline(count, 2h) AS sparkline

Now add this to the end of that search and you will see what the guts of your sparkline really is:

| map search="| makeresults | eval sparkdata=$sparkline$"

You wll see output like this:

_time                 sparkdata
2018-10-18 20:33:01 ##__SPARKLINE__##,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0

Now that you have exposed the guts, you can tack on something like this:

| rex field=sparkdata mode=sed "s/^[^,]+,//"
| eval sparkdata=split(sparkdata, ",")
| eval mvcount=mvcount(sparkdata)

| eval firstHalf=mvindex(sparkdata, 0, floor(mvcount/2))
| eval firstHalfCountNonZero = mvcount(mvfilter(firstHalf>0))

| eval lastHalf=mvindex(sparkdata, ceiling(mvcount/2), mvcount)
| eval lastHalfCountNonZero = mvcount(mvfilter(lastHalf>0))

| eval health=case((firstHalfCountNonZero==0 AND lastHalfCountNonZero==0), "DEAD",
                   (firstHalfCountNonZero==0 AND lastHalfCountNonZero>0),  "Starting",
                   (firstHalfCountNonZero>0  AND lastHalfCountNonZero==0), "Stopping",
                   true(),                                                 "ALIVE")

View solution in original post

mstjohn_splunk
Splunk Employee
Splunk Employee

hi @robertlynch2020 ,

Did the answer below solve your problem? If so, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help ya. Thanks for posting!

0 Karma

mstjohn_splunk
Splunk Employee
Splunk Employee

hi @robertlynch2020

I'm glad to see that you are using the Karma bounty feature! However, it won't work if you don't engage with the user trying to answer your question. Please approve the question below so the user can receive their Karma points. Or, if the solution didn't help you, please explain why so that they — or someone else — can.

Thanks for posting!

robertlynch2020
Motivator

sure - i have pushed back and awaiting replay

woodcock
Esteemed Legend

This is NOT a complete answer but it should give you enough to work with to craft your own. Here is a run-anywhere search that generates an "ALIVE" sparklkine (set TimePicker to All time😞

| gentimes start=1/1/17 end=10/1/18 increment=1d
| rename starttime AS _time
| stats sparkline(count, 2h) AS sparkline

Now add this to the end of that search and you will see what the guts of your sparkline really is:

| map search="| makeresults | eval sparkdata=$sparkline$"

You wll see output like this:

_time                 sparkdata
2018-10-18 20:33:01 ##__SPARKLINE__##,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0

Now that you have exposed the guts, you can tack on something like this:

| rex field=sparkdata mode=sed "s/^[^,]+,//"
| eval sparkdata=split(sparkdata, ",")
| eval mvcount=mvcount(sparkdata)

| eval firstHalf=mvindex(sparkdata, 0, floor(mvcount/2))
| eval firstHalfCountNonZero = mvcount(mvfilter(firstHalf>0))

| eval lastHalf=mvindex(sparkdata, ceiling(mvcount/2), mvcount)
| eval lastHalfCountNonZero = mvcount(mvfilter(lastHalf>0))

| eval health=case((firstHalfCountNonZero==0 AND lastHalfCountNonZero==0), "DEAD",
                   (firstHalfCountNonZero==0 AND lastHalfCountNonZero>0),  "Starting",
                   (firstHalfCountNonZero>0  AND lastHalfCountNonZero==0), "Stopping",
                   true(),                                                 "ALIVE")

robertlynch2020
Motivator

Hi - Just to come back - i reused this answer again for new issues, seriously great answer 🙂 - Hope to see you at the .cof 2021 🙂

 

The below 2 isnull commands were needed to get it going

| eval firstHalfCountNonZero = mvcount(mvfilter(firstHalf>0))
| eval firstHalfCountNonZero=if(isnull(firstHalfCountNonZero),0,firstHalfCountNonZero)
| eval lastHalf=mvindex(sparkdata, ceiling(mvcount/2), mvcount)
| eval lastHalfCountNonZero = mvcount(mvfilter(lastHalf>0))

 

 

 

0 Karma

robertlynch2020
Motivator

Mr Woodcock - good to get your help 🙂

I had to update the main question as i cant post images in this replay correctly.

So i tried your solution, but i think the "MAP" command can't be real time (I did put it into a one Relative window and it works, but i loose the sparkline update, unless i get the window to refresh every X seconds - This could be an option).

In this case i was looking for a 1 minute real time window, with three columns.

Service_name Sparkline Status

I got it down to this, so is it possible to reconstruct the sparkline to be visual again?

If so, i might be able to use a 1 minute search and refresh ever 10 seconds, to give it the feeling of real time?

Output i have:

source health sparkdata_copy
LAUNCHER.MXMLC.COLLATERAL.ASSIGN # hp548srv.fr.murex.com-54039 ALIVE 0,1,0,0,1,0,0
LAUNCHERALL # hp548srv.fr.murex.com-58085 ALIVE 0,0,1,0,1,0,0

My search:

 index=jmx sourcetype=jmx host="hp548srv.fr.murex.com:9080" jvmDescription="*" mbean_domain="murex" 
 | search source = *\=SubAgent* 
 | search source = *lid* 
 | rex field=source "^.*installationcode=(?.*),subagent-name=(?.*)" 
 | table _time source Launcher Machine_Name 
 | eval source = Launcher." # ".Machine_Name 
 | stats sparkline(count, 10s) AS sparkline by source 
 | map search="| makeresults | eval sparkdata=$sparkline$ | eval source=$source$" 
 | rex field=sparkdata mode=sed "s/^[^,]+,//" | eval sparkdata_copy=sparkdata
 | eval sparkdata=split(sparkdata, ",") 
 | eval mvcount=mvcount(sparkdata) 
 | eval firstHalf=mvindex(sparkdata, 0, floor(mvcount/2)) 
 | eval firstHalfCountNonZero = mvcount(mvfilter(firstHalf>0)) 
 | eval lastHalf=mvindex(sparkdata, ceiling(mvcount/2), mvcount) 
 | eval lastHalfCountNonZero = mvcount(mvfilter(lastHalf>0)) 
 | eval health=case((firstHalfCountNonZero==0 AND lastHalfCountNonZero==0), "DEAD",
     (firstHalfCountNonZero==0 AND lastHalfCountNonZero>0), "Starting",
     (firstHalfCountNonZero>0 AND lastHalfCountNonZero==0), "Stopping",
     true(), "ALIVE") 
 | table source health sparkdata_copy
0 Karma

robertlynch2020
Motivator

HI

I am marking this answers as accepted as it had the core of what i needed.

Thanks
Woodcock

0 Karma

woodcock
Esteemed Legend

@robertlynch2020, there you go again trying to ask something impossible. You'll have to work harder to stump me! I am starting to think that you are just making this stuff up to challenge me and that you can't possibly have a real-world use for it!

0 Karma

robertlynch2020
Motivator

Mr Woodcock - good to get your help 🙂

I had to update the main question as i cant post images in this replay correctly.

So i tried your solution, but i think the "MAP" command can't be real time (I did put it into a one Relative window and it works, but i loose the sparkline update, unless i get the window to refresh every X seconds - This could be an option).

In this case i was looking for a 1 minute real time window, with three columns.

Service_name Sparkline Status

I got it down to this, so is it possible to reconstruct the sparkline to be visual again?

If so, i might be able to use a 1 minute search and refresh ever 10 seconds, to give it the feeling of real time?

Output i have

source health sparkdata_copy
LAUNCHER.MXMLC.COLLATERAL.ASSIGN # hp548srv.fr.murex.com-54039 ALIVE 0,1,0,0,1,0,0
LAUNCHERALL # hp548srv.fr.murex.com-58085 ALIVE 0,0,1,0,1,0,0

index=jmx sourcetype=jmx host="hp548srv.fr.murex.com:9080" jvmDescription="*" mbean_domain="murex" 
| search source = *\=SubAgent* 
| search source = *lid* 
| rex field=source "^.*installationcode=(?<Launcher>.*),subagent-name=(?<Machine_Name>.*)" 
| table _time source Launcher Machine_Name 
| eval source = Launcher." # ".Machine_Name 
| stats sparkline(count, 10s) AS sparkline by source 
| map search="| makeresults | eval sparkdata=$sparkline$ | eval source=$source$" 
| rex field=sparkdata mode=sed "s/^[^,]+,//" | eval sparkdata_copy=sparkdata
| eval sparkdata=split(sparkdata, ",") 
| eval mvcount=mvcount(sparkdata) 
| eval firstHalf=mvindex(sparkdata, 0, floor(mvcount/2)) 
| eval firstHalfCountNonZero = mvcount(mvfilter(firstHalf>0)) 
| eval lastHalf=mvindex(sparkdata, ceiling(mvcount/2), mvcount) 
| eval lastHalfCountNonZero = mvcount(mvfilter(lastHalf>0)) 
| eval health=case((firstHalfCountNonZero==0 AND lastHalfCountNonZero==0), "DEAD",
    (firstHalfCountNonZero==0 AND lastHalfCountNonZero>0), "Starting",
    (firstHalfCountNonZero>0 AND lastHalfCountNonZero==0), "Stopping",
    true(), "ALIVE") 
| table source health sparkdata_copy
0 Karma

robertlynch2020
Motivator

Ha, that is funny 🙂

Thanks for the answer. I will try and get it in-today and i will get back with some replay.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...