Can I create a threat intelligence lookup that automatically updates the list of known, bad ip addresses from threat intel websites?

[b_bunch]

I'd like to be able to create lookups of known bad ip addresses (SANS, BOGON, etc) and have the lookups update automatically twice each day. I would then compare netwrok traffic and ip addresses that are visited by users with the ip addresses on the lookups.

I don't have Splunk ES and don't plan to have it. I'd like to be able to leverage Splunk Security Essentials or Cisco Security apps if possible. If not, then it will have to be built from scratch.

Has anybody done this?

[adonio · Accepted solution]

yes,

you can use your favorite search engine and look for automate download from STIX | SANS | Any other Threat Intel
pick your favorite script and use it to download your lists, download and define the lists as lookups.
it is suggested in couple of answers (this portal) that instead of updating the lookup via script (in splunk) you can monitor new lists (every period of time) and run a scheduled search periodically to populate the lookups with new values

or something little bit more advances like @woodcock answer here: https://answers.splunk.com/answers/320873/how-to-automatically-upload-csv-files-to-splunk-mo.html
as for the app development, iirc, the Cisco Security Suite does not have any queries that are set to run against a Threat Intelligence lookup, so you will probably have to do it yourself ... maybe create a macro that has the comparison you want with lookup or inputlookup command and apply to existing searches.

lastly, you can use this app:
https://splunkbase.splunk.com/app/1723/#/overview
i like it, hope you will too

hope it helps