I need to combine a normal search for 24 hr period with all events and a subsearch on threshold based event where it should query for a certain type of event exceeding a count of 3 in a hour for a host
i ran the below command provided by martin-mueller in earlier thread
index=server earliest=-24h | append [search index=server event-type=high mem-ultilzation | stats count by hostname | where NOT event-type="high mem-utilization" OR count > 3
It does provide the host which exceeded the threshold, but the count provided for the event with the threshold value is incorrect.
It gives the value as 1 with a flat sparkline, when there were 5 actually occurrences in an hour
I need the count to be displayed as 5 and not as 1
Can someone please help in martin's absence
if you look at this run everywhere command:
index=_internal earliest=-24h source="*metrics.log" | bucket _time span=10min | stats count(eval(max(kb) >= 200)) AS myCount by _time, series, host, kb | where myCount > 6 AND NOT series="summary"
does this provide a result you expect?
The search runs over the last 24 hours, builds
_time buckets of 10 minutes, counts how many times a series had more then 200 kb throughput per 10 minutes, filters out
series="summary" and also results which have less than a count of 6 (6 times a 10min bucket makes up one hour).
Try this, maybe you have to adapt the field names..but, this will point you towards the solution. I don't know if this is a copy/paste answer.
index=server | bucket _time span=1h | stats count(eval(event-type="high mem-ultilzation")) AS hi-men-count count(eval(event-type!="high mem-ultilzation")) AS other-count by _time, event-type, host | search (event-type="high mem-ultilzation" AND hi-mem-count>="3") OR (NOT event-type="high mem-ultilzation" AND other-count>="0") | eval count=if(other-count=="0", hi-mem-count, other-count) | table event-type, host, count
look, @martin_mueller did provide a way to go by using a
subsearch and I did show you a way without using one. If you don't want to use any of them I suggest you to start here http://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/WelcometotheSearchTutorial
Yes martin_mueller did provide a way using subsearch and it gives the right information but the event count and sparkline for the threshold event was incorrect .
Though they were 5 events for the host , it came as 1 event with a flat sparkline instead of coming as 5.just need that to be corrected
It is an eventtype which occurs frequently on several hosts , I only need the hosts where the event occurs more than 3 times in an hour. All other events in 24hr period need to remain as they are and no conditions need to be applied for then and need the results for both searches in a single table
here is a sample table result that i expect, as you can see only server4 is the only host with highmemutil higher than 3 occurances in an hour.
Eventype hostname Count
Diskspacefull server1 2
highmemutlization server4 5
Networkutlizationhigh server2 20
eventtype5 server3 5
Eventype is a field in your events/data and not the
eventtype search command from Splunk, right? The Splunk
eventtype search command is like a symonym for a search string; for example
eventtype=error translates to the search
error OR fatal like in the docs http://docs.splunk.com/Documentation/Splunk/6.2.0/Admin/eventtypesconf