Splunk Search

How to compare 2 lookups and highlight any changes ?



Once a month we receive a file via email that we manually upload to Splunk as a lookup CSV file.  The current process is to delete the old file and to upload the new one, keeping the same file name. The existing reports use this file without any issues.

There is now a requirement to compare the current file with the previous version and highlight if any values have been added or removed (the columns stay the same).

Initially I wanted to use the "inputlookup" and "collect" commands to output the data into an index and then build a search to compare the data based on the ingest time, effectively comparing the 2 files. However, I`m getting the following error: "The lookup table 'test.csv' requires a .csv or KV store lookup definition."

The file actually exists and it`s located in "/opt/splunk/etc/apps/test_app/lookups/test.csv"
The lookup definition also exists: "test_LD"

I suspect this is caused by the size of the lookup file (approx. 36 MB) and wanted to ask for suggestions or workarounds ?

Many thanks.

Labels (1)
Tags (1)
0 Karma
Get Updates on the Splunk Community!

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Using the Splunk Threat Research Team’s Latest Security Content

REGISTER HERE Tech Talk | Security Edition Did you know the Splunk Threat Research Team regularly releases ...