topic Re: Which of these searches is the best way to filter (by index, by source, or both)? in Splunk Search

Which of these searches is the best way to filter (by index, by source, or both)?

driekhof — Mon, 13 Mar 2017 21:05:27 GMT

Which of these would be the most efficient/fast/best way to start filtering for a search?

index=foo | ...

source="/var/log/bar/baz.log" | ...

index=foo source="/var/log/bar/baz.log" | ...

We're going to have an index that will have several **/*.log sources, each with similar but unique data formats. We'll always know the data source and index for these queries. I'm wondering the best way start my queries.

Re: Which of these searches is the best way to filter (by index, by source, or both)?

pradeepkumarg — Mon, 13 Mar 2017 23:36:31 GMT

index=foo source="/var/log/bar/baz.log" | ...

http://docs.splunk.com/Documentation/Splunk/6.5.2/Search/Writebettersearches

From the documentation

Restrict your search to the specific host, index, source, source type, or Splunk server whenever possible. Read more about using fields in your searches in the next section.

Re: Which of these searches is the best way to filter (by index, by source, or both)?

driekhof — Tue, 14 Mar 2017 00:17:02 GMT

That still isn't clear to me whether specifying both helps any over just specifying the most specific which would be source in my case. I was thinking Splunk might already know that this source is only in this index and optimize it, or already index the sources. I guess I'd have to profile using just the source vs the index and the source to be sure. But thanks for the info.

Re: Which of these searches is the best way to filter (by index, by source, or both)?

DalJeanis — Tue, 14 Mar 2017 01:53:00 GMT

In this case, having the extra data is certainly not going to hurt, but really, you ALWAYS want to specify the index, because then splunk does not have to look ANYWHERE ELSE. Giving it the source as well as that helps it narrow further.

Splunk would have figured out - almost certainly, after a glance at the summary stats - that there were none of that source anywhere else, by checking all the other indexes. But why make it go to even that meager effort?

Re: Which of these searches is the best way to filter (by index, by source, or both)?

driekhof — Tue, 14 Mar 2017 15:53:27 GMT

Ok, makes sense. I was just being paranoid about writing the shortest, clearest most concise query possible. And wanted to make sure specifying both source and index wouldn't cause Splunk to do extra work.

Re: Which of these searches is the best way to filter (by index, by source, or both)?

DalJeanis — Tue, 14 Mar 2017 19:39:30 GMT

Good goal. In this case quite the reverse, I think. If you can limit the search to a single index, or a limited set of them, then you'll (in theory) save splunk a slight bit of time in the search parsing. Overall run time is unlikely to be affected much, again, in my somewhat limited experience.