Splunk Search

how to use scripted input for refreshing lookup data?

desi
New Member

I have two files test1.csv and test2.csv. I indexed them in Splunk and then use them as lookup. These two files are refreshed everyday with updated data. What i want to do is refresh my lookups with new data in csv files. Here is what i came up with and put in refresh.bat files.

     generatetest1.csv
     generatetest2.csv
    ./splunk stop
    ./splunk clean eventdata -index test1_index -f
    ./splunk clean eventdata -index test2_index -f
    ./splunk start
    ./splunk add oneshot "C:\downloads\proto_data\csv\test1.csv" -sourcetype csv -index test1_index -rename-source test1  -auth admin:changeme
    ./splunk search "index=test1_index | outputlookup test1lookup.csv" -auth admin:changeme

./splunk add oneshot C:\downloads\proto_data\csv\test2.csv -sourcetype csv -index test2_index -rename-source test2  -auth admin:changeme
./splunk search "index=test2_index | outputlookup test2lookup.csv" -auth admin:changeme

I have two questions:

  1. is this the right way to do?
  2. if yes, how can i modify above script such that instead of calling generatetest1.csv and generatetest2.csv and creating test1.csv and test2.csv i can use scripted input and refresh my lookups.

thanks

0 Karma

melting
Splunk Employee
Splunk Employee

So there is actually a lookup search cmd which will use a csv for this purpose. If that doesn't work you can actually use a scripted lookup. Take a look at the docs. OR this blog post.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Rounding off the Splunk Dashboard Contest

What does a contest-winning Splunk dashboard look like? In this case, it isn't in a browser tab at all. It ...

A Four Part Event Series: AI + Observability: AI Agents, LLMs, Apps, & Infrastructure

AI + Observability: AI Agents, LLMs, Apps, & Infrastructure The rapid evolution of artificial intelligence ...