Getting Data In
Highlighted

Why are my automatic lookups not working?

Engager

Hey Splunk, long time lurker, first time poster.

I am attempting to perform an automatic CIDR lookup from a CSV file on a specific sourcetype. I can manually perform the lookup and get data back, but can't figure out what is wrong with my props.conf configuration for automatic results.

I appreciate any advice provided. This app is running in Splunk 6.6.3 in a Search Head Cluster.

props.conf
[rfc5424_syslog]
LOOKUP-check = IP_Ranges ip_range AS host OUTPUT range_name

transforms.conf
[IP_Ranges]
filename = ips.csv
match_type = CIDR(ip_range)
fields_list = ip_range, range_name

ips.csv
ip_range,range_name
10.0.0.0/8,"US Generic One"
10.10.10.0/24,"US Generic Two"

When I perform the following search, I see the expected results

sourcetype=rfc5424_syslog | head 20 | lookup IP_Ranges ip_range AS host OUTPUT range_name | table host, range_name

When I perform the following search, I am not seeing range_name fields added automatically

sourcetype=rfc5424_syslog
0 Karma
Highlighted

Re: Why are my automatic lookups not working?

Splunk Employee
Splunk Employee

Missing some required settings per https://answers.splunk.com/answers/305211/how-to-match-an-ip-address-from-a-lookup-table-of.html

[IPRanges]
min
matches = 1
defaultmatch = NONE
match
type = CIDR(cidr_range)

0 Karma
Highlighted

Re: Why are my automatic lookups not working?

Engager

Thanks for the reply.

I tried the minmatches and defaultmatch fields; unfortunately, that is not resolving it.

What is interesting though is defaultmatch = NONE should fill the field with NONE if their is no CIDR match; but I'm not getting this nor any values back for the iprange or range_name.

This suggests the lookup in transforms.conf is not being executed at all?

I appreciate the assistance!

0 Karma
Highlighted

Re: Why are my automatic lookups not working?

Engager

Diving in, I found that Splunk is associating the lookup to the sourcetype and is showing it in the search.log. I'm not sure what this means in terms of why its not working though.

$ /opt/splunk/bin/splunk cmd btool props list rfc5424_syslog --debug | grep -i lookup
/opt/splunk/etc/apps/XYZ/default/props.conf   LOOKUP-check = IP_Ranges ip_range AS host OUTPUT range_name

Job Inspector -> search.log

08-30-2018 17:52:18.092 INFO  LookupOperator - Using cidr matching for field 'ip_range' in lookup table 'IP_Ranges'
08-30-2018 17:52:18.092 INFO  LookupOperator - Loading lookup table='IP_Ranges', file size=82, modtime=1535640790
0 Karma
Highlighted

Re: Why are my automatic lookups not working?

Splunk Employee
Splunk Employee

These configs worked for me. Can you provide sample data?

props.conf
[iptest]
LOOKUP-check = IPranges iprange AS host OUTPUT range_name

[IPranges]
filename = ips.csv
min
matches = 1
defaultmatch = NONE
match
type = CIDR(iprange)
fields
list = iprange, rangename

ips.csv
iprange,rangename
10.0.0.0/8,"US Generic One"
10.10.10.0/24,"US Generic Two"

0 Karma
Highlighted

Re: Why are my automatic lookups not working?

Engager

Thank you for the help on this! I am emailing sample data directly.

For anyone reading this, I will update the solution once we find one.

0 Karma
Highlighted

Re: Why are my automatic lookups not working?

New Member

Hi.. did you ever find the solution for this? Facing the same issue, when running query manually it is working but nothing when used in automatic lookup

0 Karma
Highlighted

Re: Why are my automatic lookups not working?

Splunk Employee
Splunk Employee

Have you checked your permissions?

0 Karma
Highlighted

Re: Why are my automatic lookups not working?

Engager

Wanted to provide some feedback on this process. We never did figure out why it wasn't working in 6.6.3, I suspect it was the way our object permissions were setup. I got this working using the exact same code in 7.0.5. I don't think this is a bug in 6.6.3, there was a lot of funky configuration going on in the environment.

Sorry I can't be more specific than this with the configuration settings. But I do want to extend my appreciation to tprzelomiec for his assistance.

0 Karma