Why am I unable to output lookup field when the field is not a Splunk field?

[becksyboy]

Hi All,

I have an issue which i am unable to resolve. I have a lookup with two columns:

Process_Command_Line, score

Under 'Process_Command_Line', the values are wild carded e.g. *net user*.

The 'score' column has an arbitrary numerical value added.

The SPL is working with the 'Process_Command_Line' wild carded values and we only see events relevant to the Lookup values, but I cannot get the score value to be also visible. Is there something fundamentally incorrect with the SPL i am using:

index=wineventlog

source="WinEventLog:*" EventCode=4688 Process_Command_Line!="" user!="-" user="*123serviceProd*"

[| inputlookup SuspiciousDiscoveryActivity.csv | fields Process_Command_Line]

| dedup Process_Command_Line

| lookup SuspiciousDiscoveryActivity.csv Process_Command_Line as Process_Command_Line OUTPUTNEW score

thanks.

[PickleRick]

Your search in general is higly suboptimal. Most importantly, if you have any other option, do not use search terms beginning with wildcards. It makes Splunk have process every single raw event which obviously is not very efficient. Also, try to not use stuff like "user!=-".

I'm not 100% sure if match type is honored when you're calling lookup by filename, not by definition name.

[PickleRick · Accepted solution]

OK. I tested it and now I'm pretty sure that if you call lookup just by csv name it will not use the lookup settings defined for the lookup based on that lookup file.

So if I defined a simple one-row lookup table like this:

host	match
whate*	true

saved it as wildcard_test.csv then defined a wildcard_test lookup with match type WILDCARD(host)

then run

| makeresults 
| eval host="whatever" 
| lookup wildcard_test host output match as match_lookup
| lookup wildcard_test.csv host output match as match_csv

I get a result of "true" only in the match_lookup field which matches by the lookup definition. The field match_csv which comes from the lookup based on just the csv field is not matched.

[becksyboy]

Thanks for taking the time to look at this. I will re-check to see what we are doing wrong compared to your example.

[becksyboy]

Thanks again, this fixed my issue. I had actually had the same config as you, but my wildcard match condition syntax was incorrect, I had

WILDCARD(Process_Command_Line)(score)

When it should have been just

WILDCARD(Process_Command_Line)

[ITWhisperer]

Have you defined the lookup to use WILDCARD lookups?

[becksyboy]

Yes we have this defined and the wildcards for the fields seem to be working for the 'Process_Command_Line' fields.