topic Re: Unable to output Lookup field when the field is not a Splunk Field. in Splunk Search

Why am I unable to output lookup field when the field is not a Splunk field?

becksyboy — Tue, 18 Apr 2023 07:54:49 GMT

Hi All,

I have an issue which i am unable to resolve. I have a lookup with two columns:

Process_Command_Line, score

Under 'Process_Command_Line', the values are wild carded e.g. *net user*.

The 'score' column has an arbitrary numerical value added.

The SPL is working with the 'Process_Command_Line' wild carded values and we only see events relevant to the Lookup values, but I cannot get the score value to be also visible. Is there something fundamentally incorrect with the SPL i am using:

index=wineventlog

source="WinEventLog:*" EventCode=4688 Process_Command_Line!="" user!="-" user="*123serviceProd*"

[| inputlookup SuspiciousDiscoveryActivity.csv | fields Process_Command_Line]

| dedup Process_Command_Line

| lookup SuspiciousDiscoveryActivity.csv Process_Command_Line as Process_Command_Line OUTPUTNEW score

thanks.

Re: Unable to output Lookup field when the field is not a Splunk Field.

ITWhisperer — Mon, 17 Apr 2023 09:52:02 GMT

Have you defined the lookup to use WILDCARD lookups?

Re: Unable to output Lookup field when the field is not a Splunk Field.

becksyboy — Mon, 17 Apr 2023 10:28:04 GMT

Yes we have this defined and the wildcards for the fields seem to be working for the 'Process_Command_Line' fields.

Re: Unable to output Lookup field when the field is not a Splunk Field.

PickleRick — Mon, 17 Apr 2023 10:44:32 GMT

Your search in general is higly suboptimal. Most importantly, if you have any other option, do not use search terms beginning with wildcards. It makes Splunk have process every single raw event which obviously is not very efficient. Also, try to not use stuff like "user!=-".

I'm not 100% sure if match type is honored when you're calling lookup by filename, not by definition name.

Re: Unable to output Lookup field when the field is not a Splunk Field.

PickleRick — Mon, 17 Apr 2023 13:08:23 GMT

OK. I tested it and now I'm pretty sure that if you call lookup just by csv name it will not use the lookup settings defined for the lookup based on that lookup file.

So if I defined a simple one-row lookup table like this:

host	match
whate*	true

saved it as wildcard_test.csv then defined a wildcard_test lookup with match type WILDCARD(host)

then run

| makeresults 
| eval host="whatever" 
| lookup wildcard_test host output match as match_lookup
| lookup wildcard_test.csv host output match as match_csv

I get a result of "true" only in the match_lookup field which matches by the lookup definition. The field match_csv which comes from the lookup based on just the csv field is not matched.

Re: Unable to output Lookup field when the field is not a Splunk Field.

becksyboy — Mon, 17 Apr 2023 13:27:29 GMT

Thanks for taking the time to look at this. I will re-check to see what we are doing wrong compared to your example.

Re: Unable to output Lookup field when the field is not a Splunk Field.

becksyboy — Mon, 17 Apr 2023 14:34:54 GMT

Thanks again, this fixed my issue. I had actually had the same config as you, but my wildcard match condition syntax was incorrect, I had

WILDCARD(Process_Command_Line)(score)

When it should have been just

WILDCARD(Process_Command_Line)