topic Re: correlating events within a given time window in Splunk Search

correlating events within a given time window

sushil909 — Mon, 22 Jul 2013 11:12:52 GMT

Hi,
I have a file containing events in the format given below
Time system parameter value

12jun2013:14:00:00 system1 memoryusage 12345221233
12jun2013:14:00:00 system1 userprocesses 129

I have created my own custom source type. I am able to make splunk parse the data according to the fields.
I want to be able to run queries like
whenever memoryusage > 10000000 show the userprocesses within that time window(1sec)

How can i run this kind of query ?

Re: correlating events within a given time window

linu1988 — Mon, 28 Sep 2020 14:24:24 GMT

Extract the data into a field named Mem_count. You can use rex / UI field extraction. Thanks.

Sourcetype=_Name "memoryusage"| where Mem_count>10000000

Re: correlating events within a given time window

sushil909 — Mon, 22 Jul 2013 12:23:14 GMT

this would only display the 'memoryusage' events.Based on this condition i want to display the userprocesses events that may have occured some time prior (eg with 5 secs) to the memoryusage event

Re: correlating events within a given time window

linu1988 — Mon, 28 Sep 2020 14:24:27 GMT

the above query will give you the event containing the Process at the same time when the memory usage is high. You can also remove the %s parameter if you are okay with comparing minute wise.

Re: correlating events within a given time window

jameshgibson — Mon, 22 Jul 2013 15:16:09 GMT

Try something like:

sourcetype=whatever (memoryusage OR userprocesses) | rex field=_raw ".*memoryusage (?P< memoryusage>[0-9]+)" | rex field=_raw ".*userprocess (?P< userprocess>[0-9]+)"  | transaction _time maxspan=1s | search memoryusage>10000000 |  table memoryusage userprocess

formatting is a bit messed up so the < word> should really be <word>

Re: correlating events within a given time window

linu1988 — Mon, 22 Jul 2013 15:41:39 GMT

Hey James,
Great analysis, but without join how the Table will show different value belonging to separate events? I faced the same in my query to get the userprocess value..

Re: correlating events within a given time window

jameshgibson — Mon, 22 Jul 2013 15:55:19 GMT

the transaction command joins all the events for a particular second in a single event. So you should have 1 memoryusage and several userprocess per event, so no need to use joins/stats/etc. Give it a go anyway 🙂

Re: correlating events within a given time window

linu1988 — Mon, 22 Jul 2013 16:09:43 GMT

I tried it on sample event, i didn't get it. Let the actual person give it a go 🙂

Re: correlating events within a given time window

sushil909 — Tue, 23 Jul 2013 07:57:44 GMT

Thanks James...the solution works perfectly
only issue i see is that having multiple regular expression slows down the search. Since i already know the format of the event, the individual field, isn't there a way to avoid regex. For example a way to specify that whenever the parameter="memoryusage" memoryuse=value

Re: correlating events within a given time window

jameshgibson — Tue, 23 Jul 2013 08:17:51 GMT

if the log file contained paramater=value then Splunk should parse out the fields automagically for you yes. If you can't change the log format then it may be worth setting up some field extractions.

Re: correlating events within a given time window

sushil909 — Tue, 23 Jul 2013 11:08:29 GMT

Removing regex from James answer since I had field extractions in place:
sourcetype=whatever (memoryusage OR userprocesses) | eval memoryusage=case(parameter=="memoryusage",value)|eval userprocess=case(parameter=="userprocess",value)|transaction _time maxspan=1s | search memoryusage>10000000 | table memoryusage userprocess