Splunk Search

How do you get the value from a tabular event for alerting?

raj_mpl
Path Finder

Hi my log event will be in a tabular format like below

program status Group Lag Time

ABC RUNNING process1 00:03:05 00:00:04

ABC RUNNING process2 00:06:20 00:00:02

Now I want to write an alert condition when Lag is greater than 30 minutes . How do I achieve this ?

Thank you

Tags (2)
0 Karma

sdchakraborty
Contributor

Hi @raj_mpl ,

You need to tweak the multikv extracttion using the multikv.conf file. code below,

[demo_mkv]
header.start = "program"
header.linecount = 1
header.tokens = _tokenize_, -1," "
body.tokens = _tokenize_,-1, " "

You need to place this conf file in local/default folder in your app as multikv.conf. Ans restart splunk. I indexed the data in main index and the below query working for me. Then you can use your own logic on lag field.

index=main 
|  multikv conf=demo_mkv

I have discussed the same stuff below,

https://youtu.be/8kWgDVZZ0GQ

Sid

0 Karma

raj_mpl
Path Finder

Hi @sdchakraborty , Thanks for your reply on this
I need to do this using search head only ., Gone through your video about multikv its worth and good stuff
So when I fire this sourcetype=mysourcetype| multikv forceheader=2 , I am getting two fields named as Lag and Time . I believe Splunk extracted the filed name of Lag Point as Lag and Time point as Time .
Now help me with a query to build an alert to check when Lag OR Time is greater than 15 minutes

Thank you

0 Karma

raj_mpl
Path Finder

So I developed a Query like below

sourcetype=mysourcetype| multikv forceheader=2 |convert dur2sec(Lag) AS Lag_in_secs |convert dur2sec(Time) AS Lag|table Lag_in_secs, Lag |where Lag_in_secs>900 OR Lag>900

Please correct me if anything wrong here .

0 Karma

sdchakraborty
Contributor

This query looks good.
Sid

0 Karma

raj_mpl
Path Finder

Thanks Sid

0 Karma

whrg
Motivator

The convert dur2sec() function is what you are looking for:

your base search
| convert dur2sec(Lag) AS Lag_in_secs
| where Lag_in_secs>30*60
0 Karma

raj_mpl
Path Finder

Hi @whrg , thanks for your reply on this the first row that I mentioned is actually contained program status Group (lag point), (Time point)
Like below

program status Group Lag point Time point
ABC RUNNING process1 00:03:05 00:00:04
ABC RUNNING process2 00:06:20 00:00:02

So now please let me know what would be the command to split them both , I need to write condition on both lag point and Time point

0 Karma

whrg
Motivator

I'm not sure I understand. What do you mean by splitting them both? Do you want the alert to trigger when either Lag point or Time point exceeds 30 minutes?

0 Karma

raj_mpl
Path Finder

Yes absolutely , and a single event itself contains all the 3 rows in a tabular format .. I want to make 1st row as fields (program,stats,group,lap point,Time point)

0 Karma

whrg
Motivator

So your event is multiline and you are only interested in the "process1" line?

Check out this field extraction:

| makeresults count=1 | eval _raw="program status Group Lag point Time point
ABC RUNNING process1 00:03:05 00:00:04
ABC RUNNING process2 00:06:20 00:00:02"
| rex field=_raw "(?<program>\S+)\s+(?<status>\S+)\s+(?<group>\S+)\s+(?<lag_point>\d+:\d+:\d+)\s+(?<time_point>\d+:\d+:\d+)"

You might be better off indexing your logs as CSV files. This way, the fields are automatically extracted.

0 Karma

raj_mpl
Path Finder

Iam trying to achieve using mulikv command bro like

sourcetype = mydata | multikv forceheader=2| ......

Something like above query I need , it's a tabular data and Iam interested in "lag point' and "time point" .. which Iam not able to extract as fields

0 Karma

raj_mpl
Path Finder

Hi @whrg

Developed the query like below

sourcetype=mysourcetype| multikv forceheader=2 |convert dur2sec(Lag) AS Lag_in_secs |convert dur2sec(Time) AS Lag|table Lag_in_secs, Lag |where Lag_in_secs>900 OR Lag>900

Please correct me if anything wrong here .

whrg
Motivator

I think it needs to be: multikv forceheader=1

0 Karma

raj_mpl
Path Finder

Yes , My event will start with a timestamp and some other information in first line
so multikv forceheader=2 , worked for me 🙂

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...