Yes, the lib subdirectory of apps is not replicated for distributed search. Also, this app refers to the absolute path $SPLUNK_HOME/etc/apps/maps/lib/... , which isn't where replicated apps are placed for distributed search. Since the lib directory is large (over 100MB), I wouldn't recommend that the app author make the app replicate fully (as that will cause search slowdown when replication takes place). Instead I'd probably change the search to include the | localop command before the | geoip command to force the command to run on the search head rather than the indexers. ... View more

I'll assume that you've already extracted the relevant fields and have good time extraction in place. You can achieve this by either using streamstats to compute the differences between subsequent runs for each project or by dedup and stats to find the most recent two runs for each project. I'll provide the dedup solution since it seems a bit simpler to me. ... | dedup 2 Project | stats first(FailedTests) as current_failed last(FailedTests) as previous_failed by Project | where current_failed > previous_failed Provided that the search is run over a long enough time range, it will find all projects where the most recent number of failures is more than the previous recorded number of failures. You can then configure your alert to trigger when the number of events is greater than zero. The biggest problem here is that the alert will keep on firing until we have the same or fewer failures for all projects. Perhaps that's desirable. If not, you could use a lookup table to store the previous number of failures per Project. Then you'll have exactly one line in an alert per increase. A search like this will work: ... | stats first(FailedTests) as current_failed by Project | lookup failure_count.csv Project OUTPUT current_failed as past_failed | outputlookup failure_count.csv | where current_failed > past_failed Here we compute the most recent failure count by project, then we look up the previous failure count (called current_failed in the lookup), then we save our revised table and finally filter out those projects that have a past failure and a higher FailedTests than before. ... View more

I tried this as well and encountered the same problem. We'll track it down and see the root cause, but there's an easy workaround that's preferable, especially if you're putting this search command into a separate app. The active_directory python module appears to be just a single .py file. If you put this file in $SPLUNK_HOME/etc/apps/<app>/bin/ , you should be able to import it from your search command. This technique should work with any pure python module (that doesn't require native C/C++ code to be compiled). You can see an example of this in the geoip app on splunkbase: http://www.splunkbase.com/apps/All/4.x/Add-On/app:Geo+Location+Lookup+Script. In that particular case a directory is placed in the app's bin directory. ... View more

This configuration should work. I just tested it on 4.1.3 with the following in inputs.conf (to check lguinn's hypothesis): [monitor:///Users/ssorkin/tail] [batch:///Users/ssorkin/tail/sinkhole] move_policy = sinkhole If I run: ssorkin$ date >> tail/sinkhole/sinkhole.log The file is indexed and deleted from that directory. When you say that the file isn't consumed, do you mean that it's not indexed, not deleted or either. Does the user that Splunk is run as have sufficient permissions to read the file and remove them from the directory? ... View more

You cannot use setcap to grant the capability to splunkd or python in any meaningful way. Processes in linux that are run with escalated privileges cannot use a LD_LIBRARY_PATH which is essential. Your best bet is one of the suggestions from http://stackoverflow.com/questions/413807/is-there-a-way-for-non-root-processes-to-bind-to-privileged-ports-1024-on-li. ... View more