Splunk Search
Highlighted

How to update a lookup periodically?

Explorer

Hi All,

I'm wondering what would be the best way to download the latest CSV from http://cyberthreatalliance.org/cryptowall-dashboard.html

This site has information on the latest Cryptowall information (URL, IPs, Hashes, etc). I'd like to download the csv maybe once a month and update/replace the existing lookup. What would be the best way to do that? Also, would this cause any issues?

0 Karma
Highlighted

Re: How to update a lookup periodically?

SplunkTrust
SplunkTrust

Your question is too general to answer for splunk. However, you can update your lookup files using a scheduler.

Run a scheduled script every month and download the details to lookup location of splunk , for eg: in linux use a script to run a curl command. It's recommended to take a back up of your previous file before updating it.

In case you want the data to be indexed instead of lookup, use splunk's input method with custom scripts.

View solution in original post

Highlighted

Re: How to update a lookup periodically?

Splunk Employee
Splunk Employee

Once you know the location of the csv lookup on disk $SPLUNK_HOME/etc/apps/myapp/lookups/mylookup.csv
you can :

  • create a script to replace the file. It can be done while splunk is running.
  • or index the new csv file, and schedule a splunk search to
    • 1) return the results (in the correct field order)
    • 2) use the command "| outputlookup " to overwrite the existing lookup with the results.

see http://docs.splunk.com/Documentation/Splunk/6.3.2/SearchReference/Outputlookup

0 Karma
Highlighted

Re: How to update a lookup periodically?

Esteemed Legend

You need the GetWatchList app:
https://splunkbase.splunk.com/app/635/