Splunk Search

Inaccurate results from a lookup

jambajuice
Communicator

I have a lookup table that contains CVSS vulnerability metrics. The fields are as follows:

"_time","cve_id",score,"access_vector","access_complexity",authentication,"integrity_impact","availability_impact","confidentiality_impact","vuln_product"

The vuln_product field is multivalued. Sometimes events may have 100+ items in the vuln_product field. Its data looks like the following:

"videolan:vlc_media_player:0.2.82
videolan:vlc_media_player:0.2.83
videolan:vlc_media_player:0.2.80"

The lookup table is about 11 MB in size. When I perform a search with about 50,000 results and I do a lookup on the cve_id and output the rest of those fields, data from the "vuln_product" field is showing up in other fields. I've double-checked the lookup table and the data looks clean. I've compared the entries in the lookup table with events that were or were not displaying properly and I can't see any difference in the data.

What might be the cause of Splunk not successfully getting events out of the lookup table?

Thx.

Craig

Tags (1)

woodcock
Esteemed Legend

My suspicion is that your lookup table has multiple key value entries. Assuming the first field is the only key field, check for key uniqueness like this:

awk -F, '{print $1}' ${SPLUNK_HOME}/etc/MyApp/lookups/MyLookupFile.csv | uniq -d
0 Karma

jambajuice
Communicator

I also tried creating two separate lookup tables. One contains the all of the fields except for the big, multi-valued "vuln_product" field. I created another lookup table that only has cve_id and vuln_product fields. If I do a lookup on the first table and then a lookup on the second table, everything is fine. But if I try and make a lookup on a single, big table, the data gets mashed up.

Is this a bug?

0 Karma

jambajuice
Communicator

I tried splitting the lookup table into two 18,000 row (approximately) tables. When I performed the lookup on either table, the results were fine. If I combined the tables into a single one, data from the multi-valued field were still showing up in other fields.

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...