After some frustrating results I believe it's finally working. I wanted to share the script so that others don't go through this same frustration. Turns out modeling the external_lookup which is used as 'dnslookup' in searches provides some context. But now I've explicitly called out all of the options I hope. 1st Splunk lookups are fed using a 2 column csv structure and expect to have data returned in csv structure. So thats why we use 'field1' as the source of our request to the external script and 'field2' as the response back to Splunk (which is initially empty until the script fills the data). 2nd I've placed data strings for all of the results so someone doesn't get nothing back from the external lookup and can't tell if it's an error or not. 3rd SOURCE file "test_output.py" #!/usr/bin/env python ##### # # Code Stub - Splunk lookup external/scripted # # ######## # Usage # # Edit the permissions for this file to be the owner of splunk with full execute and read # placing it in the main splunk search app # $SPLUNK_HOME$/etc/system/bin/ path # # chown splunkuser:splunkgroup thisfile.py # chmod a+xr thisfile.py # # Edit the main search app's transforms.conf file # $SPLUNK_HOME$/etc/system/local/transforms.conf # Add this stanza #[test_output] #external_cmd = test_output.py field1 field2 #fields_list = field1,field2 # # Where 'test_output' is the name you want to use to call for example below # mysearch | lookup test_output field1 # and field1, field2 are the expected field names from splunk # # # Python libraries import csv import sys import socket ##### Take an Action on field1 data def takeactiononfield1data(ip): try: hostname, aliaslist, ipaddrlist = socket.gethostbyaddr(ip) return hostname except: return 'NoFQDN' ##### Take action on field2 data def takeactiononfield2data(host): try: #Do something hostname, aliaslist, ipaddrlist = socket.gethostbyname(host) return ipaddrlist except: return ['NO_IPs_Returned'] ##### Main function def main(): #Check input if len(sys.argv) != 3: print "Usage: python thisfile.py [field1] [field2]" sys.exit(1) # Input field1 = sys.argv[1] field2 = sys.argv[2] infile = sys.stdin outfile = sys.stdout # Splunk build header and expected 2 field, columns data r = csv.DictReader(infile) header = r.fieldnames w = csv.DictWriter(outfile, fieldnames=r.fieldnames) w.writeheader() #Do something with the fields for result in r: # if both fields have value write out if result[field1] and result[field2]: w.writerow(result) # if only field1 provide field2 elif result[field1]: # Static Just put string data #result[field2] = "NoField2_data" # Dynamic result[field2] = takeactiononfield1data(result[field1]) w.writerow(result) # if only field2 provide field1 elif result[field2]: #Static Just put string data result[field1] = "NoField1_data" # Do something like run a function #result[field1] = takeaction(result[field2]) if result[field1]: w.writerow(result) ##### Call Main function main() ##### exit cleanly if needed sys.exit() ... View more

I think I at least forced the collections to function by dumping a working splice example instance into this search head. Using the mongoDB from another instance I was able to 'mongorestore' the entire collection with a number of different types of IOCs from iocbucket and others. Performing this action allowed me to have the full splice mongodb instance functioning at least from the backend. At this point Splice appears to be functioning with around 50 sample IOC's inside of the mongoDB instance. As well the UI finally has the data needed for the tables and no errors are being logged into the splunkd log file anymore. So the addition of a functioning mongodb with data inside appears to have fixed my initial problem. As for why I'm using Splice vs ES; I'm using Splice for IOC collection from 3rd parties and the native TAXII/STYX/CYBOX support for quick dashboard design and alerting/reporting functions. This site also has deployed ES and will be leveraging some of the Splice additions into ES for added Endpoint integration. However, I can also use Splice and monitor for IOCs outside of ES, which can be a jacked up deployment that takes time away from monitoring for IOCs while being setup. Splice (without mongoDB issues) - up and running with sample IOCs < 4 hours to production ES (without any hiccups) - up and running within about a week often longer Thanks again for the mongo help ... View more

Thank you for the quick response; however I understand that the supporting scripts inside of Splice create the collections. But what I was looking for is there a specific script (bin/splice/database.py for example) that could be invoked directly to force creation of the collections? In this instance case the MongoDB is on the same search head as a new ES app deployment so I created an entirely new MongoDB instance for Splice with a new local port and database paths. Example, Splice MongoDB instance Listening port: 27018/tcp DBPath=/data/db (taken from MongoDB documentation directly for binary builds) Execution command: /usr/local/mongodb/bin/mongo --port=27018 Invoked as user: root (just to show that the permissions aren't the issue) On the Splunk side the Splice app has been deleted/removed using (splunk app remove SA-Splice) the approved Splunk commands. The Splunk daemon is then restarted and the SA-Splice application re-installed and configured using the application mongodb path. 'mongodb://localhost:27018/splice' This is the stage that should be creating the collections for the application but without a specific script to force install into mongo or for a log file to check what's erroring out it's impossible to determine where the app or permissions might be failing. Also my example IOC's are from iocbucket.com; I found that to be a great example to show the power of Splice. My example IOC's are pulled down via the iocbucket RSS feed then decode and store the IOC files on the search head for ingestion into Splice. ... View more