SOLVED - Splunk Search Command, Regex, and OR Operator

[fzuazo]

Greetings all,

I'm trying to search inside a lookup table and I need to use a search command follow by an OR and regex

I need the regex to match anything in the lookup table and not just the two fields before it.

Below is some sample SPL, I know it won't work this way but I'm including it to give an idea of what I'm trying to accomplish.

| inputlookup data_source.csv
| fillnull value=MISSING
| search (count=MISSING AND percent=MISSING) OR regex "[^0-9a-zA-Z\-\._,]"

Thanks in advance for the help, I really appreciate it.

[yuanliu]

First, you want to familiarize yourself with where command and how it differs from search command.  As @ITWhisperer said, search operates on _raw field.  Because inputlookup does not produce raw events, you need to specify which field or fields from data_source.csv to apply that regex.  Suppose all you want to do is to match a field named somefield, your search can be simply:

| inputlookup data_source.csv
| where (isnull(count) AND isnull(percent)) OR match(somefield, "[^0-9a-zA-Z\-\._,]")

Here, there is no need to fillnull because isnull function test the condition without a spurious assignment.

Now, if you want to apply that regex to every field from this lookup, the following should work but that's really not what Splunk is designed to do.

| inputlookup data_source.csv
| foreach *
    [eval allfields = if(isnull(allfields), "", allfields) . <<FIELD>>]
| where (isnull(count) AND isnull(percent)) OR match(allfields, "[^0-9a-zA-Z\-\._,]")

[ITWhisperer]

The search command and regex command by default work on the _raw field. This is normally present in the events in your index. Since your events are coming from a lookup, it is unlikely that you have a _raw field, which means you need to specify a field for the regex command to filter on.

Can you rewrite your filter requirement such that it can be applied to fields returned by your inputlookup?

[fzuazo]

Thanks for the input.

I can definitely do that but I need to make sure that the regex searches are chained with ORs to the previous searches.

[ITWhisperer · Accepted solution]

Try something like this

| inputlookup data_source.csv
| fillnull value="MISSING"
| where (count="MISSING" AND percent="MISSING") OR match(count, "[^0-9a-zA-Z\-\._,]") OR match(percent, "[^0-9a-zA-Z\-\._,]")

[fzuazo]

Thank you !