I am building a custom streaming search command using the Python SDK and the instructions laid out here. So far, I'm just testing the basic process of trying to get the command to run from within Splunk (i.e. the command itself doesn't do anything yet). However, I'm running into trouble simply importing the SDK libraries shown in the example. Here is a stack trace from the job inspector: I copied the splunklib folder from the SDK into my app's bin directory, and below is the streaming command's code so far. Anyone have an idea what's wrong here? I'm not sure what "No section: 'handlers'" would mean in this context. import sys from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators @Configuration() class ipasnCommand(StreamingCommand): def stream(self, events): # Put your event transformation code here pass dispatch(ipasnCommand, sys.argv, sys.stdin, sys.stdout, __name__) ... View more

This presentation (towards the end) shows the Splunk Python SDK being used to build a custom search command. Is installation of the Python SDK required to build custom search commands? I need to build a custom streaming command for a Splunk instance with limited external internet access. If the Python SDK is essential, is there another way to install it other than using pip? I'd also like to see two examples of custom command protocols V1 vs V2. I see that V2 boasts performance improvements, and I am trying to analyze some existing code for an add-on to figure out if it is using V1 or V2. Thanks for the help! ... View more

I want to find out more about how custom commands work in Splunk Apps (specifically for geoip lookup type apps). I've perused the code in several apps now (GeoASN, geoip, SecKit, etc.), and I'm trying to find the most performant way to query a MaxMind Database and map client IP's to Autonomous System Numbers (ASN's). I keep seeing this kind of thing in each app's transforms.conf file: [command_name] external_cmd = command_name.py fields_list = field1 field2 etc I'm assuming this is how Splunk knows what data to pipe to which external command... But what I don't know is the real process by which Splunk invokes those commands and passes results back to the eventset. Here's why I need to know: If you have to do a lot of MaxMind lookups on a dataset, it's a lot faster if you can cache some results in memory. So, if Splunk is calling out to your add-on application's MaxMind lookup script separately for each lookup, a lot of performance is lost. So, what I ask is how do external commands like this really work? And what kind of flexibility is there in how they are invoked? Would it be possible to keep a script running so you can cache MaxMind data while you run all the lookups in a streaming-type manner? As always, thanks for any input you have ... View more

It appears that my use of the REST API is somehow causing a leading pipe to be stripped before an inputcsv command. I have this python search string: "| inputcsv scale_med_validation_data | apply fastflux_model | where 'predicted(is_attack)' = 1 | eval t = now()+3600*1 | eval report_hour=strftime(t, "%H") | eval report_date=strftime(t, "%m/%d/%Y") | tail 50 | collect index=fastflux_summary" This works as desired when entered manually through the web interface. However, when submitted through the REST API, the jobs screen shows the search query missing the leading pipe: "inputcsv scale_med_validation_data | apply fastflux_model | where 'predicted(is_attack)' = 1 | eval t = now()+3600*1 | eval report_hour=strftime(t, "%H") | eval report_date=strftime(t, "%m/%d/%Y") | tail 50 | collect index=fastflux_summary" Naturally, this causes the inputcsv to fail, and so none of the REST API jobs succeed. Why might the leading pipe not be making it through here? ... View more

Splunk's command types page is missing a few functions, including accum. I would like to know if accum is a centralized streaming command, distributable streaming command, or none of the above. Essentially, I need to know if it can run on indexers, so that I don't have to bring the whole event set back to the search head before computing totals. For my use case, this is crucial for developing a scalable solution. One comment at the bottom of the accum page suggests that it functions similarly to streamstats . . . which might imply that it is a centralized streaming command. If so, that would be disappointing: what I'm really looking for is a way to offload the computation of some counts/totals to a group of distributed indexers. I am using the Machine Learning Toolkit and working with some large DNS datasets, and it appears that every command type except for distributable streaming commands cannot run on indexers. I am worried that Splunk may lack an important cluster computing capability: being able to compute intermediary statistics on separate nodes (e.g. map-reduce). It sounds like stats and other transforming commands really only run on the search head . . . meaning the entire event set has to be pulled from the distributed indexers into a single node before computing counts, totals, averages, etc. From a cluster computing perspective, this lack of parallelism is concerning . . . and if there is no way to compute subtotals on indexers or otherwise parallelize the summarization process before data reaches the search head, it would seem that a custom big data platform is still the only answer for high-volume machine learning tasks like this one. I hope I'm wrong about this, or that I missed something in the documentation. Please let me know if you have any other information. ... View more