Dear Splunkers,
If I could get an answer on how do I find which HEC token is causing authentication failures (num_of_auth_failures=1) from _introspection logs, will very much helpful.
I'm using below query to find the errors, but how do I pin point which is causing the issue?
index=_introspection component=TERM(HttpEventCollector) "data.series"=TERM(http_event_collector) (data.num_of_auth_failures=1 OR data.num_of_requests_to_disabled_token=1 OR data.num_of_requests_to_incorrect_url=1)
Thanks in Advance.
This is a really old post but I had the same problem. A search query that appears to be helping me find these problems is:
index=_internal sourcetype=splunkd log_level=ERROR component=HttpInputDataHandler
The results are imperfect because they don't exactly match what's shown in the authentication failures, but in my case, it appears the errors are being caused by a source that is sending in blank/missing tokens.