Splunk Search

How to compare 2 lookups and highlight any changes ?



Once a month we receive a file via email that we manually upload to Splunk as a lookup CSV file.  The current process is to delete the old file and to upload the new one, keeping the same file name. The existing reports use this file without any issues.

There is now a requirement to compare the current file with the previous version and highlight if any values have been added or removed (the columns stay the same).

Initially I wanted to use the "inputlookup" and "collect" commands to output the data into an index and then build a search to compare the data based on the ingest time, effectively comparing the 2 files. However, I`m getting the following error: "The lookup table 'test.csv' requires a .csv or KV store lookup definition."

The file actually exists and it`s located in "/opt/splunk/etc/apps/test_app/lookups/test.csv"
The lookup definition also exists: "test_LD"

I suspect this is caused by the size of the lookup file (approx. 36 MB) and wanted to ask for suggestions or workarounds ?

Many thanks.

Labels (1)
Tags (1)
0 Karma
Get Updates on the Splunk Community!

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...

Want to Reduce Costs, Mitigate Risk, Improve Performance, or Increase Efficiencies? ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...