Splunk Search

How to update a lookup periodically?

CYBR_AH
Explorer

Hi All,

I'm wondering what would be the best way to download the latest CSV from http://cyberthreatalliance.org/cryptowall-dashboard.html

This site has information on the latest Cryptowall information (URL, IPs, Hashes, etc). I'd like to download the csv maybe once a month and update/replace the existing lookup. What would be the best way to do that? Also, would this cause any issues?

0 Karma
1 Solution

renjith_nair
SplunkTrust
SplunkTrust

Your question is too general to answer for splunk. However, you can update your lookup files using a scheduler.

Run a scheduled script every month and download the details to lookup location of splunk , for eg: in linux use a script to run a curl command. It's recommended to take a back up of your previous file before updating it.

In case you want the data to be indexed instead of lookup, use splunk's input method with custom scripts.

Happy Splunking!

View solution in original post

woodcock
Esteemed Legend

You need the GetWatchList app:
https://splunkbase.splunk.com/app/635/

yannK
Splunk Employee
Splunk Employee

Once you know the location of the csv lookup on disk $SPLUNK_HOME/etc/apps/myapp/lookups/mylookup.csv
you can :

  • create a script to replace the file. It can be done while splunk is running.
  • or index the new csv file, and schedule a splunk search to
    • 1) return the results (in the correct field order)
    • 2) use the command "| outputlookup " to overwrite the existing lookup with the results.

see http://docs.splunk.com/Documentation/Splunk/6.3.2/SearchReference/Outputlookup

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Your question is too general to answer for splunk. However, you can update your lookup files using a scheduler.

Run a scheduled script every month and download the details to lookup location of splunk , for eg: in linux use a script to run a curl command. It's recommended to take a back up of your previous file before updating it.

In case you want the data to be indexed instead of lookup, use splunk's input method with custom scripts.

Happy Splunking!
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...