I have a customer who is trying to query the Splunk REST API using an established AD service account, which has the same privileges as individual accounts. When he tries to query Splunk, however, he gets "Unauthorized". Is it possible to query the REST API with non-local accounts?
Yea its possible. In fact, all the web activity basically runs REST calls under the hood. It's likely the user is trying to use their unix credentials instead of their splunk credentials. Or maybe the system they are calling to doesn't have the ldap configured? Make sure the ldap strategy lives on everything (except the forwarders if you want) so the same credentials are valid when the search is delegated from search head to indexers.
I always set up my ldap config in an app that I distribute to EVERYTHING because I don't want anyone logging on with local credentials and making a mess of things. I want only credible audit-able accounts logging into the system.
I vaguely recall experiences before I did this where users defined locally only on the SH were failing because the indexer knew nothing of that user. My memory sucks so I may have conflated experiences 😞
Oh, if you don't already have it, make sure splunkweb is OFF on the indexers. If you need to make config changes, those should go through the master node. Not only will that open up some resources back to indexer activities (because splunkweb is off) but it blocks users from accidentally navigating to the web UI and getting mixed up.
Either way, you shouldn't have issues with the non local account thang. If that is persisting, we might collaborate offline to see what's up.