I have a dashboard that analysts use to quickly triage events. Within this dashboard I have several panels that separate events based on correlation across different sensors, callbacks, and miscellaneous events such as binaries, certain types of web infections, etc. I need to add hash reputation to events triggered from binaries. I have a custom Python script that accepts a hash value, submits it to a reputation cloud, and parses the response to determine reputation. We receive hundreds of hashes per hour from certain sensors, all of which get passed to this reputation cloud for reputation, and could cause a lookup table to become large. I've also had issues in the past with kv stores seeming to roll or lose data, though I won't rule out a configuration issue since I was using kv stores to implement event acknowledgments. My question is - is there a way to use props / transforms / custom scripts to flag on events that have a hash, pass that hash to my custom script, and add a new reputation field to the event with the response value prior to indexing? I know this could potentially slow indexing and searching. Otherwise, is there a reliable way to use a lookup table or kv store to do this, understanding that my panels have a mixture of event types and that the table could grow large with millions of entries? Beyond displaying reputation data within the dashboard, we also need the ability to search only for events with a certain reputation and the ability to create metrics on this reputation data for leadership, so any type of rolling or aging off with lookup tables or kv stores isn't acceptable. Thanks. ... View more

I have a Python script that queries an external system for reputation data based on a hash. What I would like to do is for every event being ingested by Splunk, execute this script prior to index and add a field to the raw event containing the hash reputation. The events are a combination of XML and CEF syslog currently, and currently there are only two fields that could contain a hash value. Ideally what would happen, probably on the HFW, would be: This event comes in: <title>Event 1</title> <hash>123ac898aaa809f90a</hash> This event goes out for Splunk to index: <title>Event 1</title> <hash>123ac898aaa809f90a</hash> <reputation>Known Bad</reputation This would allow me to query for all hashes with a certain reputation in Splunk and perform other operations without having to perform a lookup. My thought was that this could be accomplished by props.conf and transforms.conf calling an external script, similar to an external lookup, but I'm not sure how to configure Splunk to do this without using a lookup table and to add the field to the event prior to ingest? ... View more