I have a file containing events in the format given below
Time system parameter value
12jun2013:14:00:00 system1 memoryusage 12345221233
12jun2013:14:00:00 system1 userprocesses 129
I have created my own custom source type. I am able to make splunk parse the data according to the fields.
I want to be able to run queries like
whenever memoryusage > 10000000 show the userprocesses within that time window(1sec)
How can i run this kind of query ?
Try something like:
sourcetype=whatever (memoryusage OR userprocesses) | rex field=_raw ".*memoryusage (?P< memoryusage>[0-9]+)" | rex field=_raw ".*userprocess (?P< userprocess>[0-9]+)" | transaction _time maxspan=1s | search memoryusage>10000000 | table memoryusage userprocess
formatting is a bit messed up so the < word> should really be <word>
Removing regex from James answer since I had field extractions in place:
sourcetype=whatever (memoryusage OR userprocesses) | eval memoryusage=case(parameter=="memoryusage",value)|eval userprocess=case(parameter=="userprocess",value)|transaction _time maxspan=1s | search memoryusage>10000000 | table memoryusage userprocess
if the log file contained paramater=value then Splunk should parse out the fields automagically for you yes. If you can't change the log format then it may be worth setting up some field extractions.
Thanks James...the solution works perfectly
only issue i see is that having multiple regular expression slows down the search. Since i already know the format of the event, the individual field, isn't there a way to avoid regex. For example a way to specify that whenever the parameter="memoryusage" memoryuse=value
I tried it on sample event, i didn't get it. Let the actual person give it a go 🙂
the transaction command joins all the events for a particular second in a single event. So you should have 1 memoryusage and several userprocess per event, so no need to use joins/stats/etc. Give it a go anyway 🙂
Great analysis, but without join how the Table will show different value belonging to separate events? I faced the same in my query to get the userprocess value..
sourcetype=_Name "userprocesses"|eval a=strptime(_time,"%d:%m:%y %H:%M:%S")|fields a [|search index=main sourcetype=_Name "memoryusage"|where MCount >10000000|eval a=strptime(_time,"%d:%m:%y %H:%M:%S")|fields a]
the above query will give you the event containing the Process at the same time when the memory usage is high. You can also remove the %s parameter if you are okay with comparing minute wise.
this would only display the 'memoryusage' events.Based on this condition i want to display the userprocesses events that may have occured some time prior (eg with 5 secs) to the memoryusage event
Extract the data into a field named Mem_count. You can use rex / UI field extraction. Thanks.
Sourcetype=_Name "memoryusage"| where Mem_count>10000000