Splunk Search

how to use scripted input for refreshing lookup data?

desi
New Member

I have two files test1.csv and test2.csv. I indexed them in Splunk and then use them as lookup. These two files are refreshed everyday with updated data. What i want to do is refresh my lookups with new data in csv files. Here is what i came up with and put in refresh.bat files.

     generatetest1.csv
     generatetest2.csv
    ./splunk stop
    ./splunk clean eventdata -index test1_index -f
    ./splunk clean eventdata -index test2_index -f
    ./splunk start
    ./splunk add oneshot "C:\downloads\proto_data\csv\test1.csv" -sourcetype csv -index test1_index -rename-source test1  -auth admin:changeme
    ./splunk search "index=test1_index | outputlookup test1lookup.csv" -auth admin:changeme

./splunk add oneshot C:\downloads\proto_data\csv\test2.csv -sourcetype csv -index test2_index -rename-source test2  -auth admin:changeme
./splunk search "index=test2_index | outputlookup test2lookup.csv" -auth admin:changeme

I have two questions:

  1. is this the right way to do?
  2. if yes, how can i modify above script such that instead of calling generatetest1.csv and generatetest2.csv and creating test1.csv and test2.csv i can use scripted input and refresh my lookups.

thanks

0 Karma

melting
Splunk Employee
Splunk Employee

So there is actually a lookup search cmd which will use a csv for this purpose. If that doesn't work you can actually use a scripted lookup. Take a look at the docs. OR this blog post.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Free Professional Services for .conf26 Attendees

This year at .conf26, we are doing something a little different. We are bringing the best minds from ...

Defend at Machine Speed: Your Guide to Security Sessions at .conf26

Splunk .conf26   With threats moving at machine speed and attack surfaces expanding across hybrid ...

Where Innovation Takes Flight: The Splunk4Aviation Flight Sim Lands at .conf26

If you hear someone at .conf26 shouting "gear down, GEAR DOWN" across the show floor, you have found us.  The ...