Is it possible for Splunk to natively run a search against a remote Splunk REST API from within a search? For example, let's say I have a search that is monitoring apache logs for suspicious behavior. The central index/search server finds three IP addresses that are behaving suspiciously. I would like Splunk to be able to run a search on the remote web server's REST API that would execute a saved search with a python script in it that adds each address to the iptables chain and reloads the rules.
Or would I just have to write the python script on the central server and call it that way?
There is no search command natively available that provides this ability. You would have to code your own Python search command to perform this for you and format its output as search results.
If you would like an example of a similar search command, you can check out the entity command in the Splunk on Splunk app, which we use to query to splunkd REST API local to where the command is executed (search-head or search peers).