Splunk Search

How would you do a computation between a single value and multi value variables?

pjdwyer
Explorer

The event s I am dealing with have multiple "instance times" to work with, I am trying to find the time difference between the event time and the instance times.

Here is an example of one event:

Jun 19 13:00:00 hostname 1.1.1.1
Alert: Inactive process 2.2.2.2 6/14/18 12:00:00 process 03
Alert: Inactive process 2.2.2.32 6/12/18 06:00:00 process 40
Alert: Inactive process 2.2.2.8 6/1/18 18:00:00 process 86

I capture the inactive processes in a mv field and the event time is given. However, I cannot seem to find a way to do the computation if the event has more than one process down.

| rex field=_raw "(?<servereventtime>\w{3}\s+(?<eventDay>\d+)\s(?<eventHour>\d{2}):(?<eventMin>\d{2}):(?<eventSec>\d{2}))\s(?<host>[\w\.-]+)\s+(?<hostip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| rex field=_raw "(?<space>\s)"
| rex field=_raw "(?<ip>\d)(?<date_down>\w{8}\s(?<downYear>\d+)\S(?<downMon>\d+)\S(?<downDay>\d+)\s(?<downHour>\d+)\S(?<downMin>\d+)\S(?<downSec>\d+))"
| rex field=_raw "(?<ip>\w+\S\w{13}\S\w{3})(?<date_down>\w{8}\s(?<downYear>\d+)\S(?<downMon>\d+)\S(?<downDay>\d+)\s(?<downHour>\d+)\S(?<downMin>\d+)\S(?<downSec>\d+))" max_match=100
| rex field=_raw "(?<ip>\d+\S\d+\S\d+\S\d+)(?<date_down>\w{8}\s(?<downYear>\d+)\S(?<downMon>\d+)\S(?<downDay>\d+)\s(?<downHour>\d+)\S(?<downMin>\d+)\S(?<downSec>\d+))" max_match=100
| eval eventYear=strftime(_time,"%Y")
| eval eventMon=strftime(_time,"%m")
| eval secDown=case(eventSec<downSec,eventSec+60-downSec,1==1,eventSec-downSec)
| eval eventMin=case(eventMin>0 AND eventSec<downSec,eventMin-1,eventSec<downSec,59,1=1,eventMin)
| eval minDown=case(eventMin<downMin,eventMin+60-downMin,1=1,eventMin-downMin)
| eval eventHour=case(eventHour>0 AND eventMin<downMin,eventHour-1,eventMin<downMin,23,1=1,eventHour)
| eval hourDown=case(eventHour<downHour,eventHour+24-downHour,1=1,eventHour-downHour)
| eval eventDay=case(eventDay>1 AND eventHour<downHour,eventDay-1,(eventMon=2 OR eventMon=4 OR eventMon=6 OR eventMon=8 OR eventMon=9 OR eventMon=11 OR eventMon=1) AND eventHour<downHour,31,(eventMon=5 OR eventMon=7 OR eventMon=10 OR eventMon=12) AND eventHour<downHour,30,eventMon=3 AND eventHour<downHour,28,1=1,eventDay)
| eval dayDown=case((eventMon=2 OR eventMon=4 OR eventMon=6 OR eventMon=8 OR eventMon=9 OR eventMon=11 OR eventMon=1) AND eventDay<downDay,eventDay+31-downDay,(eventMon=5 OR eventMon=7 OR eventMon=10 OR eventMon=12) AND eventDay<downDay,eventDay+30-downDay,eventMon=3 AND eventDay<downDay,eventDay+28-downDay,1=1,eventDay-downDay)
| eval eventMon=case(eventMon>1 AND eventDay<downDay,eventMon-1,eventDay<downDay,12,1=1,eventMon)
| eval monDown=case(eventMon<downMon,eventMon+12-downMon,1=1,eventMon-downMon)
| eval eventYear=case(eventMon<downMon,eventYear-1,1=1,eventYear)
| eval yearDown=(eventYear-downYear)
| eval time_down=mvzip(ip,mvzip(yearDown,mvzip(monDown,mvzip(dayDown,mvzip(hourDown,mvzip(minDown,mvzip(secDown,space," second(s)")," minute(s) ")," hour(s) ")," day(s) ")," month(s) ")," year(s) ")," Down for: ")
| eval ip_time_down=case(isnull(time_down),mvzip(ip,space," Down Time Unknown"),1=1,time_down)
| stats values(ip_time_down) by date_down, host, hostip

This is my current method, the math and the rex logic works. The issue is when there is more than one down process the stats show 0 values.

In the end the display should appear similar to this:

host | hostip | date_down | values(ip_time_down)
host | 1.1.1.1 | 6/14/18 12:00:00 |2.2.2.2 _ years, _ months, _ days, _ hours, _ mins, _ secs
| | 6/12/18 6:00:00 |2.2.2.32 _ years, _ months, _ days, _ hours, _ mins, _ secs
| | 6/1/18 18:00:00 |2.2.2.8 _ years, _ months, _ days, _ hours, _ mins, _ secs

0 Karma
1 Solution

woodcock
Esteemed Legend

Like this:

| makeresults | eval _raw="Jun 19 13:00:00 hostname 1.1.1.1
Alert: Inactive process 2.2.2.2 6/14/18 12:00:00 process 03
Alert: Inactive process 2.2.2.32 6/12/18 06:00:00 process 40
Alert: Inactive process 2.2.2.8 6/1/18 18:00:00 process 86"
| streamstats count AS _serial

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| rex "(?<servereventtime>\w{3}\s+(?<eventDay>\d+)\s(?<eventHour>\d{2}):(?<eventMin>\d{2}):(?<eventSec>\d{2}))\s(?<host>[\w\.-]+)\s+(?<hostip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| eval _time=strptime(servereventtime, "%b %d %H:%M:%S")
| rex max_match=0 "(?<Alert>Alert: [^\r\n]+)"
| rename _serial AS serial
| mvexpand Alert
| rex field=Alert max_match=0 "(?<ip>\d+\.\d+\.\d+\.\d+)\s+(?<date_down>(?<downMon>\d+)\/(?<downDay>\d+)\/(?<downYear>\d+)\s+(?<downHour>\d+):(?<downMin>\d+):(?<downSec>\d+))"
| eval date_down=strptime(date_down, "%m/%d/%y %H:%M:%S")
| eval ip_time_down = tostring(_time - date_down, "duration")
| fields _time, serial, host, hostip, ip, date_down, ip_time_down
| stats first(_time) AS _time first(host) AS host first(hostip) AS hostip list(ip) AS ip list(date_down) AS date_down list(ip_time_down) AS ip_time_down BY serial
| fieldformat date_down=strftime(date_down, "%m/%d/%y %H:%M:%S")

View solution in original post

woodcock
Esteemed Legend

Like this:

| makeresults | eval _raw="Jun 19 13:00:00 hostname 1.1.1.1
Alert: Inactive process 2.2.2.2 6/14/18 12:00:00 process 03
Alert: Inactive process 2.2.2.32 6/12/18 06:00:00 process 40
Alert: Inactive process 2.2.2.8 6/1/18 18:00:00 process 86"
| streamstats count AS _serial

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| rex "(?<servereventtime>\w{3}\s+(?<eventDay>\d+)\s(?<eventHour>\d{2}):(?<eventMin>\d{2}):(?<eventSec>\d{2}))\s(?<host>[\w\.-]+)\s+(?<hostip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| eval _time=strptime(servereventtime, "%b %d %H:%M:%S")
| rex max_match=0 "(?<Alert>Alert: [^\r\n]+)"
| rename _serial AS serial
| mvexpand Alert
| rex field=Alert max_match=0 "(?<ip>\d+\.\d+\.\d+\.\d+)\s+(?<date_down>(?<downMon>\d+)\/(?<downDay>\d+)\/(?<downYear>\d+)\s+(?<downHour>\d+):(?<downMin>\d+):(?<downSec>\d+))"
| eval date_down=strptime(date_down, "%m/%d/%y %H:%M:%S")
| eval ip_time_down = tostring(_time - date_down, "duration")
| fields _time, serial, host, hostip, ip, date_down, ip_time_down
| stats first(_time) AS _time first(host) AS host first(hostip) AS hostip list(ip) AS ip list(date_down) AS date_down list(ip_time_down) AS ip_time_down BY serial
| fieldformat date_down=strftime(date_down, "%m/%d/%y %H:%M:%S")

woodcock
Esteemed Legend

BTW, this should not be necessary. It should be valid to do math with a single-value field and a multi-value field, but it apparently is not. You should raise this as a bug with support. I understand not supporting math with 2 (more than 1) multi-valued fields, but...

0 Karma

pjdwyer
Explorer

Thank you so much! The idea to capture the alerts then expand them and work from there worked perfectly with my original method. I tried to use yours, but I think the max_match=0 on the ip and date capture was breaking it, and the time was not subtracting to give a proper difference. Thank you again!

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...