Splunk Search

How to compare 2 lookups and highlight any changes ?



Once a month we receive a file via email that we manually upload to Splunk as a lookup CSV file.  The current process is to delete the old file and to upload the new one, keeping the same file name. The existing reports use this file without any issues.

There is now a requirement to compare the current file with the previous version and highlight if any values have been added or removed (the columns stay the same).

Initially I wanted to use the "inputlookup" and "collect" commands to output the data into an index and then build a search to compare the data based on the ingest time, effectively comparing the 2 files. However, I`m getting the following error: "The lookup table 'test.csv' requires a .csv or KV store lookup definition."

The file actually exists and it`s located in "/opt/splunk/etc/apps/test_app/lookups/test.csv"
The lookup definition also exists: "test_LD"

I suspect this is caused by the size of the lookup file (approx. 36 MB) and wanted to ask for suggestions or workarounds ?

Many thanks.

Labels (1)
Tags (1)
0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...