correlating events within a given time window

[sushil909]

Hi,
I have a file containing events in the format given below
Time system parameter value

12jun2013:14:00:00 system1 memoryusage 12345221233
12jun2013:14:00:00 system1 userprocesses 129

I have created my own custom source type. I am able to make splunk parse the data according to the fields.
I want to be able to run queries like
whenever memoryusage > 10000000 show the userprocesses within that time window(1sec)

How can i run this kind of query ?

[jameshgibson]

Try something like:

sourcetype=whatever (memoryusage OR userprocesses) | rex field=_raw ".*memoryusage (?P< memoryusage>[0-9]+)" | rex field=_raw ".*userprocess (?P< userprocess>[0-9]+)"  | transaction _time maxspan=1s | search memoryusage>10000000 |  table memoryusage userprocess

formatting is a bit messed up so the < word> should really be <word>

[sushil909]

Removing regex from James answer since I had field extractions in place:
sourcetype=whatever (memoryusage OR userprocesses) | eval memoryusage=case(parameter=="memoryusage",value)|eval userprocess=case(parameter=="userprocess",value)|transaction _time maxspan=1s | search memoryusage>10000000 | table memoryusage userprocess

[jameshgibson]

if the log file contained paramater=value then Splunk should parse out the fields automagically for you yes. If you can't change the log format then it may be worth setting up some field extractions.

[sushil909]

Thanks James...the solution works perfectly
only issue i see is that having multiple regular expression slows down the search. Since i already know the format of the event, the individual field, isn't there a way to avoid regex. For example a way to specify that whenever the parameter="memoryusage" memoryuse=value

[linu1988]

I tried it on sample event, i didn't get it. Let the actual person give it a go 🙂

[jameshgibson]

the transaction command joins all the events for a particular second in a single event. So you should have 1 memoryusage and several userprocess per event, so no need to use joins/stats/etc. Give it a go anyway 🙂

[linu1988]

Hey James,
Great analysis, but without join how the Table will show different value belonging to separate events? I faced the same in my query to get the userprocess value..

[linu1988]

sourcetype=_Name "userprocesses"|eval a=strptime(_time,"%d:%m:%y %H:%M:%S")|fields a [|search index=main sourcetype=_Name "memoryusage"|where MCount >10000000|eval a=strptime(_time,"%d:%m:%y %H:%M:%S")|fields a]

the above query will give you the event containing the Process at the same time when the memory usage is high. You can also remove the %s parameter if you are okay with comparing minute wise.

[sushil909]

this would only display the 'memoryusage' events.Based on this condition i want to display the userprocesses events that may have occured some time prior (eg with 5 secs) to the memoryusage event

[linu1988]

Extract the data into a field named Mem_count. You can use rex / UI field extraction. Thanks.

Sourcetype=_Name "memoryusage"| where Mem_count>10000000