topic Using wildcards in splunk lookups, can't match second field in Splunk Search

Using wildcards in splunk lookups, can't match second field

Josh1890 — Sun, 02 Jun 2024 11:37:52 GMT

Hello, I need help with the following scenario:

Let's say I have a log source with browser traffic data, one of the available fields is malware_signature

I made a lookup table to filter the results by 10 specific malwares I'd like to be alerted on, all 10 entries have wildcards like so, with another field called classification:

malware_signature	classification
mimikatz	high

when I use inputlookup to filter the results it works well, but no matter what I tried I can't get the "classification" field to be added

works well for filtering:

[| inputlookup malware_list.csv | fields malware_signature]

classification field won't show:

[| inputlookup malware_list.csv | fields malware_signature classification]

Doesn't work:

[| inputlookup malware_list.csv | fields malware_signature]
| lookup malware_list.csv malware_signature OUTPUT classification

Clarification:

I use inputlookup for filtering the results to the logs I want to see by the malware_signature

After that I want to enrich the table with the classification field, but using the lookup command it won't catch the malware_signature with the wildcards.

Re: Using wildcards in splunk lookups, can't match second field

gcusello — Sat, 01 Jun 2024 17:20:50 GMT

Hi @Josh1890,

please try this:

<your_search> [| inputlookup malware_list.csv | rename malware_signature AS query | fields query ]

in thsi way you perform a full text search using themalware_signature field.

Ciao.

Giuseppe

Re: Using wildcards in splunk lookups, can't match second field

Josh1890 — Sat, 01 Jun 2024 17:42:26 GMT

Hey Giuseppe,

Will that allow me to add the classification field from the lookup table?

Re: Using wildcards in splunk lookups, can't match second field

yuanliu — Sun, 02 Jun 2024 06:27:43 GMT

I am confused. If you want to enrich data with classification, why use inputlookup? Just create a lookup with match_type=WILDCARD(malware_signature) if you haven't. In your third search, I see that you have defined a lookup named malware_list.csv. If so, you must have missed MATCH_TYPE. (See Create a CSV lookup definition) Then, use lookup command instead of inputlookup.

``` your search that returns malware_signature ``` | lookup malware_list.csv malware_signature | where isnotnull(classification)

Re: Using wildcards in splunk lookups, can't match second field

PickleRick — Sun, 02 Jun 2024 08:36:59 GMT

These are two different things. One thing is generating conditions using subsearch, another thing is enriching you results with a lookup.

Important thing though, generating conditions where search term has a wildcard at the beginning makes no sense performancewise. Splunk still has to read all events from the index and search them one by one. It cannot use indexed structures.

Re: Using wildcards in splunk lookups, can't match second field

gcusello — Sun, 02 Jun 2024 09:17:07 GMT

Hi @Josh1890 ,

as also @PickleRick said, it's diefferent to search and to entich, using my solution you search for the patterns contained in your lookup.

If you need to agg the classification, the only way is to use the lookup command.

Ciao.

Giuseppe

Re: Using wildcards in splunk lookups, can't match second field

Josh1890 — Sun, 02 Jun 2024 11:29:50 GMT

Hey, yes I use inputlookup for filtering the results to the logs I want to see by the malware_signature

After that I want to enrich the table with the classification field, but using the lookup command it won't catch the malware_signature with the wildcards.

Re: Using wildcards in splunk lookups, can't match second field

Josh1890 — Sun, 02 Jun 2024 11:30:35 GMT

Hey, yes I use inputlookup for filtering the results to the logs I want to see by the malware_signature

After that I want to enrich the table with the classification field, but using the lookup command it won't catch the malware_signature with the wildcards

Re: Using wildcards in splunk lookups, can't match second field

Josh1890 — Sun, 02 Jun 2024 11:48:56 GMT

You're right, but it'll run every 15 minutes for a limited amount of data, so we can suffer the performance issue

Re: Using wildcards in splunk lookups, can't match second field

PickleRick — Sun, 02 Jun 2024 12:48:32 GMT

Yes, but it makes no sense to add another layer of processing since you're gonna go through every event anyway.

So the best approach here would be to do

your basic search
| lookup enriching your data
| filter out data not matching your criteria based on lookup values

Re: Using wildcards in splunk lookups, can't match second field

yuanliu — Mon, 03 Jun 2024 03:59:41 GMT

As I said, it only means that you didn't set up wildcard matching correctly. Check your lookup setup.