topic Re: How can I get query only the last ten files from each device regardless of time range? in Splunk Search

How can I get query only the last ten files from each device regardless of time range?

peterh26 — Sun, 12 Mar 2017 08:45:31 GMT

I am looking at 10,000 devices and want to look at the last ten files each one has produced. Some will create 100 files a day, some 10 files per week, so I cannot use date, I need to be able to get a count out of the last ten files that violate specific criteria.

Re: How can I get query only the last ten files from each device regardless of time range?

jkat54 — Sun, 12 Mar 2017 17:41:50 GMT

index=yourIndexName [index=yourIndexName | dedup host | fields host | return host] | dedup 10 source

Unfortunately there is a limitation of 10,000 on subsearches I believe. Besides this next one might be exactly what you need.

index=yourIndexName | dedup 10 host source | table host source | sort 0 host

Events returned by dedup are based on search order. For historical searches, the most recent events are searched first. For real-time searches, the first events that are received are search, which are not necessarily the most recent events.

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Dedup

Re: How can I get query only the last ten files from each device regardless of time range?

peterh26 — Mon, 13 Mar 2017 02:15:51 GMT

Thank you kat54! That works great. I was using a field that goes up by one for each subsequent sample run or "file": source|stats first("Service Counter") as frst last("Service Counter") as lst by "instr_id" |eval Range=(frst - lst) |where Range<10

The problem is that I want to have all ten most recent files for each device, then perform stats on the results (essentially filter the results and find the median, mean, or certain fields, as well the count of fields out of those ten files that meet the filtering requirements. I tried doing a join to the solution you gave me and the one I had come up with (yours is much more elegant and I have found a workaround using it). I am really new to Splunk and need some more time to test one more idea, but for now, I can export the solution you provided with the fields I need to perform stats on to Excel or SAS and do the refining there. Make sense? I'm not sure how much detail to go into.

Re: How can I get query only the last ten files from each device regardless of time range?

jkat54 — Mon, 13 Mar 2017 19:59:54 GMT

The dedup 10 will give the latest 10 results assuming timestamps are correctly ordered.

Re: How can I get query only the last ten files from each device regardless of time range?

peterh26 — Mon, 20 Mar 2017 04:58:05 GMT

thanks again, this has been an incredibly helpful tool in my analysis, much appreciated 🙂