We ended up mapping api-splunk.<company>.com to a certain search head. Our concern with giving it to the users is the fact that the search head captain is not aware of searches that hit the api-splunk.<company>.com end point and also because users might abuse the access.
For one thing, I would like to set an alert for certain users that use the API and ensure that they don’t pass a certain threshold in the number of searches they invoke via the API access. How can I find out these searches given the user name?
Try this to find API users (substituting WHODIS with the username of the person you want to search for):
index=_internal sourcetype=splunkd_access NOT ( user="splunk-system-user" OR user="-") NOT clientip="127.0.0.1" user=WHODIS
| rex field=uri_path "search/jobs/(?<search_id>[^/\?]+)"
| eval search_id="'"+coalesce(search_id,id)+"'"
If this reply helps you, an upvote would be appreciated.
do you have a load balancer in front of your shc? if so, is there a reason you don't load balance the api as well?
You may be able to tie the access logs to the audit logs but I'm not positive. Not sure if there is any sort of "this is an api" search in the logs or if you have to base it on connections to the management port?