Getting Data In

Why are my automatic lookups not working?

rchurch0505
Engager

Hey Splunk, long time lurker, first time poster.

I am attempting to perform an automatic CIDR lookup from a CSV file on a specific sourcetype. I can manually perform the lookup and get data back, but can't figure out what is wrong with my props.conf configuration for automatic results.

I appreciate any advice provided. This app is running in Splunk 6.6.3 in a Search Head Cluster.

props.conf
[rfc5424_syslog]
LOOKUP-check = IP_Ranges ip_range AS host OUTPUT range_name

transforms.conf
[IP_Ranges]
filename = ips.csv
match_type = CIDR(ip_range)
fields_list = ip_range, range_name

ips.csv
ip_range,range_name
10.0.0.0/8,"US Generic One"
10.10.10.0/24,"US Generic Two"

When I perform the following search, I see the expected results

sourcetype=rfc5424_syslog | head 20 | lookup IP_Ranges ip_range AS host OUTPUT range_name | table host, range_name

When I perform the following search, I am not seeing range_name fields added automatically

sourcetype=rfc5424_syslog
0 Karma

rchurch0505
Engager

Wanted to provide some feedback on this process. We never did figure out why it wasn't working in 6.6.3, I suspect it was the way our object permissions were setup. I got this working using the exact same code in 7.0.5. I don't think this is a bug in 6.6.3, there was a lot of funky configuration going on in the environment.

Sorry I can't be more specific than this with the configuration settings. But I do want to extend my appreciation to tprzelomiec for his assistance.

0 Karma

tprzelomiec
Splunk Employee
Splunk Employee

Missing some required settings per https://answers.splunk.com/answers/305211/how-to-match-an-ip-address-from-a-lookup-table-of.html

[IP_Ranges]
min_matches = 1
default_match = NONE
match_type = CIDR(cidr_range)

0 Karma

rchurch0505
Engager

Thanks for the reply.

I tried the min_matches and default_match fields; unfortunately, that is not resolving it.

What is interesting though is default_match = NONE should fill the field with NONE if their is no CIDR match; but I'm not getting this nor any values back for the ip_range or range_name.

This suggests the lookup in transforms.conf is not being executed at all?

I appreciate the assistance!

0 Karma

rchurch0505
Engager

Diving in, I found that Splunk is associating the lookup to the sourcetype and is showing it in the search.log. I'm not sure what this means in terms of why its not working though.

$ /opt/splunk/bin/splunk cmd btool props list rfc5424_syslog --debug | grep -i lookup
/opt/splunk/etc/apps/XYZ/default/props.conf   LOOKUP-check = IP_Ranges ip_range AS host OUTPUT range_name

Job Inspector -> search.log

08-30-2018 17:52:18.092 INFO  LookupOperator - Using cidr matching for field 'ip_range' in lookup table 'IP_Ranges'
08-30-2018 17:52:18.092 INFO  LookupOperator - Loading lookup table='IP_Ranges', file size=82, modtime=1535640790
0 Karma

tprzelomiec
Splunk Employee
Splunk Employee

These configs worked for me. Can you provide sample data?

props.conf
[iptest]
LOOKUP-check = IP_ranges ip_range AS host OUTPUT range_name

[IP_ranges]
filename = ips.csv
min_matches = 1
default_match = NONE
match_type = CIDR(ip_range)
fields_list = ip_range, range_name

ips.csv
ip_range,range_name
10.0.0.0/8,"US Generic One"
10.10.10.0/24,"US Generic Two"

0 Karma

rchurch0505
Engager

Thank you for the help on this! I am emailing sample data directly.

For anyone reading this, I will update the solution once we find one.

0 Karma

payalc
New Member

Hi.. did you ever find the solution for this? Facing the same issue, when running query manually it is working but nothing when used in automatic lookup

0 Karma

tprzelomiec
Splunk Employee
Splunk Employee

Have you checked your permissions?

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...