How would you do a computation between a single value and multi value variables?

[pjdwyer]

The event s I am dealing with have multiple "instance times" to work with, I am trying to find the time difference between the event time and the instance times.

Here is an example of one event:

Jun 19 13:00:00 hostname 1.1.1.1
Alert: Inactive process 2.2.2.2 6/14/18 12:00:00 process 03
Alert: Inactive process 2.2.2.32 6/12/18 06:00:00 process 40
Alert: Inactive process 2.2.2.8 6/1/18 18:00:00 process 86

I capture the inactive processes in a mv field and the event time is given. However, I cannot seem to find a way to do the computation if the event has more than one process down.

| rex field=_raw "(?<servereventtime>\w{3}\s+(?<eventDay>\d+)\s(?<eventHour>\d{2}):(?<eventMin>\d{2}):(?<eventSec>\d{2}))\s(?<host>[\w\.-]+)\s+(?<hostip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| rex field=_raw "(?<space>\s)"
| rex field=_raw "(?<ip>\d)(?<date_down>\w{8}\s(?<downYear>\d+)\S(?<downMon>\d+)\S(?<downDay>\d+)\s(?<downHour>\d+)\S(?<downMin>\d+)\S(?<downSec>\d+))"
| rex field=_raw "(?<ip>\w+\S\w{13}\S\w{3})(?<date_down>\w{8}\s(?<downYear>\d+)\S(?<downMon>\d+)\S(?<downDay>\d+)\s(?<downHour>\d+)\S(?<downMin>\d+)\S(?<downSec>\d+))" max_match=100
| rex field=_raw "(?<ip>\d+\S\d+\S\d+\S\d+)(?<date_down>\w{8}\s(?<downYear>\d+)\S(?<downMon>\d+)\S(?<downDay>\d+)\s(?<downHour>\d+)\S(?<downMin>\d+)\S(?<downSec>\d+))" max_match=100
| eval eventYear=strftime(_time,"%Y")
| eval eventMon=strftime(_time,"%m")
| eval secDown=case(eventSec<downSec,eventSec+60-downSec,1==1,eventSec-downSec)
| eval eventMin=case(eventMin>0 AND eventSec<downSec,eventMin-1,eventSec<downSec,59,1=1,eventMin)
| eval minDown=case(eventMin<downMin,eventMin+60-downMin,1=1,eventMin-downMin)
| eval eventHour=case(eventHour>0 AND eventMin<downMin,eventHour-1,eventMin<downMin,23,1=1,eventHour)
| eval hourDown=case(eventHour<downHour,eventHour+24-downHour,1=1,eventHour-downHour)
| eval eventDay=case(eventDay>1 AND eventHour<downHour,eventDay-1,(eventMon=2 OR eventMon=4 OR eventMon=6 OR eventMon=8 OR eventMon=9 OR eventMon=11 OR eventMon=1) AND eventHour<downHour,31,(eventMon=5 OR eventMon=7 OR eventMon=10 OR eventMon=12) AND eventHour<downHour,30,eventMon=3 AND eventHour<downHour,28,1=1,eventDay)
| eval dayDown=case((eventMon=2 OR eventMon=4 OR eventMon=6 OR eventMon=8 OR eventMon=9 OR eventMon=11 OR eventMon=1) AND eventDay<downDay,eventDay+31-downDay,(eventMon=5 OR eventMon=7 OR eventMon=10 OR eventMon=12) AND eventDay<downDay,eventDay+30-downDay,eventMon=3 AND eventDay<downDay,eventDay+28-downDay,1=1,eventDay-downDay)
| eval eventMon=case(eventMon>1 AND eventDay<downDay,eventMon-1,eventDay<downDay,12,1=1,eventMon)
| eval monDown=case(eventMon<downMon,eventMon+12-downMon,1=1,eventMon-downMon)
| eval eventYear=case(eventMon<downMon,eventYear-1,1=1,eventYear)
| eval yearDown=(eventYear-downYear)
| eval time_down=mvzip(ip,mvzip(yearDown,mvzip(monDown,mvzip(dayDown,mvzip(hourDown,mvzip(minDown,mvzip(secDown,space," second(s)")," minute(s) ")," hour(s) ")," day(s) ")," month(s) ")," year(s) ")," Down for: ")
| eval ip_time_down=case(isnull(time_down),mvzip(ip,space," Down Time Unknown"),1=1,time_down)
| stats values(ip_time_down) by date_down, host, hostip

This is my current method, the math and the rex logic works. The issue is when there is more than one down process the stats show 0 values.

In the end the display should appear similar to this:

host | hostip | date_down | values(ip_time_down)
host | 1.1.1.1 | 6/14/18 12:00:00 |2.2.2.2 _ years, _ months, _ days, _ hours, _ mins, _ secs
| | 6/12/18 6:00:00 |2.2.2.32 _ years, _ months, _ days, _ hours, _ mins, _ secs
| | 6/1/18 18:00:00 |2.2.2.8 _ years, _ months, _ days, _ hours, _ mins, _ secs

[woodcock · Accepted solution]

Like this:

| makeresults | eval _raw="Jun 19 13:00:00 hostname 1.1.1.1
Alert: Inactive process 2.2.2.2 6/14/18 12:00:00 process 03
Alert: Inactive process 2.2.2.32 6/12/18 06:00:00 process 40
Alert: Inactive process 2.2.2.8 6/1/18 18:00:00 process 86"
| streamstats count AS _serial

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| rex "(?<servereventtime>\w{3}\s+(?<eventDay>\d+)\s(?<eventHour>\d{2}):(?<eventMin>\d{2}):(?<eventSec>\d{2}))\s(?<host>[\w\.-]+)\s+(?<hostip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| eval _time=strptime(servereventtime, "%b %d %H:%M:%S")
| rex max_match=0 "(?<Alert>Alert: [^\r\n]+)"
| rename _serial AS serial
| mvexpand Alert
| rex field=Alert max_match=0 "(?<ip>\d+\.\d+\.\d+\.\d+)\s+(?<date_down>(?<downMon>\d+)\/(?<downDay>\d+)\/(?<downYear>\d+)\s+(?<downHour>\d+):(?<downMin>\d+):(?<downSec>\d+))"
| eval date_down=strptime(date_down, "%m/%d/%y %H:%M:%S")
| eval ip_time_down = tostring(_time - date_down, "duration")
| fields _time, serial, host, hostip, ip, date_down, ip_time_down
| stats first(_time) AS _time first(host) AS host first(hostip) AS hostip list(ip) AS ip list(date_down) AS date_down list(ip_time_down) AS ip_time_down BY serial
| fieldformat date_down=strftime(date_down, "%m/%d/%y %H:%M:%S")

[woodcock]

BTW, this should not be necessary. It should be valid to do math with a single-value field and a multi-value field, but it apparently is not. You should raise this as a bug with support. I understand not supporting math with 2 (more than 1) multi-valued fields, but...

[pjdwyer]

Thank you so much! The idea to capture the alerts then expand them and work from there worked perfectly with my original method. I tried to use yours, but I think the max_match=0 on the ip and date capture was breaking it, and the time was not subtracting to give a proper difference. Thank you again!