Splunk Search

How to search multiple value on the same field

BunnyHop
Contributor

I have a regex that searches for different types of value on a field:

  • | regex _raw="FIELD=(value1|value2|value3)"

However this search is painfully slow. How can you perform this search on the indexed data without creating a very long search string like the below:

SEARCH FIELD="value1" OR FIELD="value2" OR FIELD="value3"

Tags (3)
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

You create the long search string. You can create the long search string indirectly via a subsearch though, which can load a file:

[ inputlookup mylist.csv | fields MYFIELDNAME | format ]

The file mylist.csv must be in the app lookups folder (probably etc/apps/search/lookups) and must be a CSV file with at least 1 column named MYFIELDNAME, one value per line. You can also generate the lookups from search results using outputlookup if that is the source of your values.

Update:

With just a little more work, you can also configure a lookup that maps MYFIELDNAME values to a "groupname", and if you then configure automatic lookups against MYFIELDNAME, then you can just add groupname="group" to the query and Splunk will automatically perform the equivalent of expanding the search string for every MYFIELDNAME value in the lookup table where groupname is mapped to group.

View solution in original post

oreoshake
Communicator

Yeah it's painfully slow because the data is filtered AFTER the search has run. I'd just write a little script that expands it out for you on your local machine. Splunk 4.0.10 recently removed the cap or OR clauses which might be good for you.

ruby:

ARGV[0].sub('|', ' or host=')

gkanapathy
Splunk Employee
Splunk Employee

You create the long search string. You can create the long search string indirectly via a subsearch though, which can load a file:

[ inputlookup mylist.csv | fields MYFIELDNAME | format ]

The file mylist.csv must be in the app lookups folder (probably etc/apps/search/lookups) and must be a CSV file with at least 1 column named MYFIELDNAME, one value per line. You can also generate the lookups from search results using outputlookup if that is the source of your values.

Update:

With just a little more work, you can also configure a lookup that maps MYFIELDNAME values to a "groupname", and if you then configure automatic lookups against MYFIELDNAME, then you can just add groupname="group" to the query and Splunk will automatically perform the equivalent of expanding the search string for every MYFIELDNAME value in the lookup table where groupname is mapped to group.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

Sorry but "local" I mean the Splunk search server, not your client workstation.

arungeorge09
Path Finder

How do I construct a query with this.?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

The external source is a file on the local machine. It will be fast.

BunnyHop
Contributor

This seems to look at values from an external source, correct? Is this more efficient?

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.