Receiving "DISABLED_DUE_TO_GRACE_PERIOD"

[mathewchase]

I have seen many questions about disabled due to licensing violation, but I applied a reset key and now I have this message during a search from the search head:

  restricting search to internal indexes only (reason: [DISABLED_DUE_TO_GRACE_PERIOD,0])

The indexing cluster appears to be functioning and able to communicate to the license server on the search head. I am still unable to get back to normal searching after a licensing violation.

Any ideas? I will call Splunk tomorrow and ask support directly and post back.

[splunkoptimus]

I had the same error I did all that @khourihan_splun  mentioned and tests were negative. to solve the problem I restarted splunkd on the indexer in question. 

[nmurillo_splunk]

Take a look at your pass4SymmKey in your server.conf.

[khourihan_splun]

We were seeing the SH return a message of DISABLED_DUE_TO_GRACE_PERIOD
This means the SH can't connect to the LM, which makes it seem like we have a DNS problem.

I experienced this error message in a Splunk 6.0.3 cluster.

Here's how I troubleshot it:

• On the indexers, check the name of the slave-master listed in the server.conf
• Check to see what IP address the slave master is resolving
• Test connectivity from indexer to LM.

If you can not connect, you have a DNS issue, and as @matthewchase above described, the work around. But I'd recommend fixing your DNS ! 🙂

Another test is to run this on the licenser endpoint:

curl -k -u admin:changeme https://localhost:8089/services/licenser/localslave

[mathewchase · Accepted solution]

So Splunk support never got back to me but I figured out the problem. The issue was indeed a communication problem between the indexing nodes and the license server. I was testing connectivity based on IP address and ports (seems reasonable — right?) All the nodes could reach the licensing server correctly. The issue was actually related to DNS failing and the indexing nodes could not resolve the license server address. I ensured the license server was entered into the hosts file and operations returned to normal.