How to create a look up table with "*&" or "any" field?

[Erfan]

Hi

I am trying to whitelist some traffic from my search. So I decided to create a look up table including src ip, dst ip, username, dst port, src zone.

for example:

src_ip	dest_ip	src_zone	dest_port	user	whitelisted
*	*	center	*	*	TRUE
172.16.20.44	13.58.90.11	*	443	Alice	TRUE
*	128.221.236.246	*	443	*	TRUE
192.168.12.03	*	*	*	*	TRUE
172.16.20.13	*	*	*	*	TRUE
192.168.26.4	*	*	*	*	TRUE
192.168.26.8	198.160.25.74	*	443	*	TRUE
192.168.26.9	198.160.25.87	*	*	*	TRUE
*	142.250.70.174	*	*	*	TRUE

but the problem is the search matches when all cells related to fields which are called in lookup command have values, but it does not match (does not whitelist) if a cell has "*" or "Any"  value.

| lookup whitelisttest.csv src_ip as src_ip dest_ip as dest_ip dest_port as dest_port user as user
| where isnull(whitelisted)

[yuanliu]

Will match_type WILDCARD help?  See Create a CSV lookup definition.

[Erfan]

Thanks for your response. 

I created a look up definition and in match type I entered:  WILDCARD(dest_ip)

I expected to see all result except traffic from 172.16.20.13:

172.16.20.13

*

*

*

*

TRUE

but it did not showed any records. I mean it remove all result not only whitelisted.

[yuanliu]

You need to give more details about your data with illustration, including an explanation of key characteristics, and illustrate the desired results.

If every event contains these four fields, src_ip, dest_ip, dest_port, and user, your lookup

| lookup whitelisttest.csv src_ip dest_ip dest_port user ``` no need to use "as" annotator if the name is the same ```​

will always return whitelisted "TRUE".  This is because in addition to the row you just quoted, you also have this row

src_ip	dest_ip	src_zone	dest_port	user	whitelisted
*	*	center	*	*	TRUE

In other words, the whild card search is working exactly as you asked.

In fact, I suggested wildcard only because you entered "*" in the table.  Splunk's lookup also supports CIDR match.  This is probably more appropriate for IP address filtering.  Just food for thought.

[Erfan]

Thank you Yuanliu

let me rephrase my question, maybe lookup is not a good solution for my problem.

Actually I want to whitelist following traffic on search of fortigate's logs:

Traffic from 172.16.20.12 to every where

traffic from 192.168.26.8 to 198.160.25.74

traffic from every where to 142.250.70.174

traffic from any to any related to a specific user (Alice)

All records include all fields.

[yuanliu]

Suitability is always defined by the data and requirements.  In the case you described, you can still use lookup to establish the desired whitelist.  For example, if you use match_type WILDCARD in every field, you can say

src_ip	dest_ip	src_zone	dest_port	user	whitelisted
172.16.20.12	*	center	*	*	TRUE
192.168.26.8	198.160.25.74	center	*	*	TRUE
*	142.250.70.174	center	*	*	TRUE
*	*	center	*	Alice	TRUE

and use the same search

| lookup whitelisttest.csv src_ip dest_ip dest_port user
| where isnull(whitelisted)

Now, if you want future ability to use CIDR for more granular control, set src_ip and dest_ip to use CIDR, and change lookup to

src_ip	dest_ip	src_zone	dest_port	user	whitelisted
172.16.20.12/32	0.0.0.0/0	center	*	*	TRUE
192.168.26.8/32	198.160.25.74/32	center	*	*	TRUE
0.0.0.0/0	142.250.70.174/32	center	*	*	TRUE
0.0.0.0/0	0.0.0.0/0	center	*	Alice	TRUE

Hope this helps.

[Erfan]

Thank you so much.

The only vague part for me is that how to create match type for wildcard on all fields in GUI version.

I think this works for CUI: 
field1,field2,field3,output_field,match_type

value1,value2,value3,output_value,*

But I do not have acccess to CUI and it is just GUI. So I defined WILDCARD(*) but did not work.

[yuanliu · Accepted solution]

You can add as many wildcard lookup fields as needed. This is the instruction from Splunk Web (GUI)

Optionally set up non-exact matching of a comma-and-space-delimited field list. Format is <match_type>(<field_name>). Available values for match_type are WILDCARD and CIDR.

No need for CUI.