topic Re: What is the most efficient way to filter search results by list? in Splunk Search

What is the most efficient way to filter search results by list?

AlexMcDuffMille — Thu, 31 Jul 2014 13:14:24 GMT

Hello,

I am looking to filter my search results by the 'UniqueID' field so that I only get results from the devices that I want to monitor.

I believe there are two ways to do this:

Search like this (make it an event type for easier reading): index=index1 (UniqueId=1 OR UniqueId=2 OR UniqueId=3 OR UniqueId=4)
Create a lookup table and use input lookup.

Does anyone know if there is a 3rd option? Or if one is more efficient than the other?

Thanks!

Re: What is the most efficient way to filter search results by list?

somesoni2 — Thu, 31 Jul 2014 13:29:14 GMT

In your second option, you're talking about filtering entries at base search level only using inputlookup right?

Means like this--

index=index1 [|inputlookup uniqueIds.csv | table UniqueId] ...| rest of the search

Re: What is the most efficient way to filter search results by list?

AlexMcDuffMille — Thu, 31 Jul 2014 13:30:00 GMT

Correct. I have not yet implemented it, but I believe that is the general gist of how to do it.

Re: What is the most efficient way to filter search results by list?

somesoni2 — Thu, 31 Jul 2014 18:42:17 GMT

I don't think there is a better approach then these two. Best place for filter is in the main search pipeline. Among these two options, first one will be little faster(I guess) as no sub searches, but may will become complex to write/read as the no of UniqueIds will increase. I would go with option two as its configurable.

Re: What is the most efficient way to filter search results by list?

martin_mueller — Fri, 01 Aug 2014 22:23:11 GMT

You're probably looking for tags: http://docs.splunk.com/Documentation/Splunk/6.1.2/Knowledge/Abouttagsandaliases

You'd define a tag monitoredDevice and configure your list of UniqueId values for that tag. Then you can search like this:

index=foo tag=monitoredDevice

Great to reuse in other searches and to be maintained in one location.

Under the hood Splunk will resolve that to the long list of OR'd values, giving you the best search performance.

Re: What is the most efficient way to filter search results by list?

martin_mueller — Fri, 01 Aug 2014 22:24:51 GMT

Eventtypes would work here as well, but in my mind you're not actually defining a type of event but rather grouping a set of UniqueId values for any type of event they may produce. Performance would be the same.
Additionally, tags let you add/remove/edit values individually while an eventtype only lets you edit a massive string. Adding or removing UniqueId values should be much less error-prone with tags.