Splunk Search

how to use scripted input for refreshing lookup data?

desi
New Member

I have two files test1.csv and test2.csv. I indexed them in Splunk and then use them as lookup. These two files are refreshed everyday with updated data. What i want to do is refresh my lookups with new data in csv files. Here is what i came up with and put in refresh.bat files.

     generatetest1.csv
     generatetest2.csv
    ./splunk stop
    ./splunk clean eventdata -index test1_index -f
    ./splunk clean eventdata -index test2_index -f
    ./splunk start
    ./splunk add oneshot "C:\downloads\proto_data\csv\test1.csv" -sourcetype csv -index test1_index -rename-source test1  -auth admin:changeme
    ./splunk search "index=test1_index | outputlookup test1lookup.csv" -auth admin:changeme

./splunk add oneshot C:\downloads\proto_data\csv\test2.csv -sourcetype csv -index test2_index -rename-source test2  -auth admin:changeme
./splunk search "index=test2_index | outputlookup test2lookup.csv" -auth admin:changeme

I have two questions:

  1. is this the right way to do?
  2. if yes, how can i modify above script such that instead of calling generatetest1.csv and generatetest2.csv and creating test1.csv and test2.csv i can use scripted input and refresh my lookups.

thanks

0 Karma

melting
Splunk Employee
Splunk Employee

So there is actually a lookup search cmd which will use a csv for this purpose. If that doesn't work you can actually use a scripted lookup. Take a look at the docs. OR this blog post.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...