Using wildcards in splunk lookups, can't match second field

[Josh1890]

Hello, I need help with the following scenario:

Let's say I have a log source with browser traffic data, one of the available fields is malware_signature

I made a lookup table to filter the results by 10 specific malwares I'd like to be alerted on, all 10 entries have wildcards like so, with another field called classification:

malware_signature	classification
*mimikatz*	high

when I use inputlookup to filter the results it works well, but no matter what I tried I can't get the "classification" field to be added

works well for filtering:

[| inputlookup malware_list.csv | fields malware_signature]

classification field won't show:

[| inputlookup malware_list.csv | fields malware_signature classification]

Doesn't work:

[| inputlookup malware_list.csv | fields malware_signature]
| lookup malware_list.csv malware_signature OUTPUT classification

Clarification:

 I use inputlookup for filtering the results to the logs I want to see by the malware_signature

After that I want to enrich the table with the classification field, but using the lookup command it won't catch the malware_signature with the wildcards.

[yuanliu]

I am confused.  If you want to enrich data with classification, why use inputlookup? Just create a lookup with match_type=WILDCARD(malware_signature) if you haven't. In your third search, I see that you have defined a lookup named malware_list.csv.  If so, you must have missed MATCH_TYPE. (See Create a CSV lookup definition) Then, use lookup command instead of inputlookup.

``` your search that returns malware_signature ```
| lookup malware_list.csv malware_signature
| where isnotnull(classification)

[Josh1890]

Hey, yes I use inputlookup for filtering the results to the logs I want to see by the malware_signature

After that I want to enrich the table with the classification field, but using the lookup command it won't catch the malware_signature with the wildcards.

[yuanliu]

As I said, it only means that you didn't set up wildcard matching correctly.  Check your lookup setup.

[gcusello]

Hi @Josh1890,

please try this:

<your_search>
[| inputlookup malware_list.csv | rename malware_signature AS query | fields query ]

in thsi way you perform a full text search using themalware_signature field.

Ciao.

Giuseppe

[Josh1890]

Hey Giuseppe,

Will that allow me to add the classification field from the lookup table?

[gcusello]

Hi @Josh1890 ,

as also @PickleRick said, it's diefferent to search and to entich, using my solution you search for the patterns contained in your lookup.

If you need to agg the classification, the only way is to use the lookup command.

Ciao.

Giuseppe

[Josh1890]

Hey, yes I use inputlookup for filtering the results to the logs I want to see by the malware_signature

After that I want to enrich the table with the classification field, but using the lookup command it won't catch the malware_signature with the wildcards

[PickleRick]

These are two different things. One thing is generating conditions using subsearch, another thing is enriching you results with a lookup.

Important thing though, generating conditions where search term has a wildcard at the beginning makes no sense performancewise. Splunk still has to read all events from the index and search them one by one. It cannot use indexed structures.

[Josh1890]

You're right, but it'll run every 15 minutes for a limited amount of data, so we can suffer the performance issue

[PickleRick]

Yes, but it makes no sense to add another layer of processing since you're gonna go through every event anyway.

So the best approach here would be to do

your basic search
| lookup enriching your data
| filter out data not matching your criteria based on lookup values