Splunk Search

Correlating Different Events and Calculating Time Difference

dw385
Explorer

I’m trying to report on the time difference between two related events. Both events are collected from Windows event log and are in the same index. They will occur on the same host but at different times and IDs.
Example event data:

Time: 3/11/2016 9:00:45 PM
EventID: 1000
Source: Backup Agent Message:
Backup Started
Host: Server1
Time: 3/12/2016 1:09:45 AM

EventID: 1001
Source: Backup Agent
Message: Backup Completed
Host: Server1

The second event can occur any time after the first, maybe in a few minutes or a few hours. I’m thinking I need to find event 1(1000) first and then look for the first occurrence of event 2 afterwards. The index would contain similar events from many hosts around the same time.
I’ve tried various attempts with joins, transactions, and subsearches and searching the documentation for suggestions with no luck. I’m really not concerned with any of the data in the event other than the time (which I would eval to a start and end time). The goal is to be able to run a report over a time span to show the start/end times for each host.

This gets me close but doesn’t scale to multiple hosts/time ranges:

index="wineventlog" SourceName="Backup Agent" (EventCode="1000" OR EventCode="1001") 
| transaction maxspan=4h startswith=(EventCode="1000") endswith=(EventCode="1001") 
| table host,duration
Tags (1)
0 Karma

somesoni2
Revered Legend

How many times these events can happen from one host in a day?

0 Karma

dw385
Explorer

Usually only one occurrence but its possible to see multiple in a day as jobs are restarted.

0 Karma

lguinn2
Legend

Try this first:

 index="wineventlog" SourceName="Backup Agent" (EventCode="1000" OR EventCode="1001") 
 | transaction host maxspan=4h startswith=(EventCode="1000") endswith=(EventCode="1001") 
 | table host,duration

All I did was add host to the transaction command. This should help, but it still may not be a good solution, if you have a long time range.
You could also try this:

 index="wineventlog" SourceName="Backup Agent" (EventCode="1000" OR EventCode="1001") 
| sort host,_time
| eval starttime=if(EventCode="1000",_time,null())
| eval endtime=if(EventCode="1001",_time,null())
| streamstats current=t window=2 global=f latest(starttime) as start latest(endtime) as end by host
| where isnotnull(endtime)
| eval duration=end-start

This solution should work with somewhat larger timeframes. I am not sure how well it will work if you have multiple backups for a single host.

0 Karma
Get Updates on the Splunk Community!

Unlock New Opportunities with Splunk Education: Explore Our Latest Courses!

At Splunk Education, we’re dedicated to providing top-tier learning experiences that cater to every skill ...

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...