https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41770#M2003 <P>have you changed your admin passwords recently and is the search head using the correct admin user of the indexer ? <BR /> I believe this refers to invalid authentication between search peer and indexer and can be setup in the search peers part of the management console.</P> Wed, 12 Oct 2011 11:57:21 GMT anttih 2011-10-12T11:57:21Z https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41769#M2002 <P>My indexers started throwing this:</P> <P>07-02-2011 04:00:58.307 -0300 WARN AdminHandler:AuthenticationHandler - Denied session token for user: splunk-system-user</P> <P>07-02-2011 04:01:05.551 -0300 WARN AdminHandler:AuthenticationHandler - Denied session token for user: splunk-system-user</P> <P>As a result, I get an error on the Search Heads stating that it has "skipped indexing of internal audit event will keep dropping events..."</P> <P>The last time I got this warning at the Search Heads it was due to the optimization taking too long, but this one is new to me, any ideas?</P> Tue, 12 Jul 2011 18:29:51 GMT https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41769#M2002 Stefan 2011-07-12T18:29:51Z https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41770#M2003 <P>have you changed your admin passwords recently and is the search head using the correct admin user of the indexer ? <BR /> I believe this refers to invalid authentication between search peer and indexer and can be setup in the search peers part of the management console.</P> Wed, 12 Oct 2011 11:57:21 GMT https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41770#M2003 anttih 2011-10-12T11:57:21Z https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41771#M2004 <P>Invalid authentication between the search peers and the search head distributing search to these peers would result in an 'Authenication Failed' type of message. This error indicates something is wrong with the token. That the search peer is sending back something in its header requesting access via an invalid token. It could also be related to time I suppose. Have you looked at splunkd_access.log to see if there are any errors in that file?</P> Mon, 14 Nov 2011 18:15:05 GMT https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41771#M2004 jbsplunk 2011-11-14T18:15:05Z https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41772#M2005 <P>It may be due to an edition of distsearch.conf without establishing trusts (indexers must trust searchhead's certificates) so you can fix this problem by following what's written in this <A href="http://docs.splunk.com/Documentation/Splunk/5.0.2/Deploy/Configuredistributedsearch#Edit_distsearch.conf">paragraph</A>.</P> Wed, 29 May 2013 09:39:11 GMT https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41772#M2005 yoho 2013-05-29T09:39:11Z https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41773#M2006 <P>Incorrect. The search head only uses the admin password of the indexer once - to send over a certificate. The certificate is used in Distributed Search from there on, so it does not matter if any password is changed on the indexer.</P> Tue, 21 Oct 2014 20:03:33 GMT https://community.splunk.com/t5/Dashboards-Visualizations/splunkd-log-Denied-session-token-for-user-splunk-system-user/m-p/41773#M2006 Jason 2014-10-21T20:03:33Z