Splunk Search

Receiving "DISABLED_DUE_TO_GRACE_PERIOD"

mathewchase
Engager

I have seen many questions about disabled due to licensing violation, but I applied a reset key and now I have this message during a search from the search head:

  restricting search to internal indexes only (reason: [DISABLED_DUE_TO_GRACE_PERIOD,0])

The indexing cluster appears to be functioning and able to communicate to the license server on the search head. I am still unable to get back to normal searching after a licensing violation.

Any ideas? I will call Splunk tomorrow and ask support directly and post back.

Labels (1)
1 Solution

mathewchase
Engager

So Splunk support never got back to me but I figured out the problem. The issue was indeed a communication problem between the indexing nodes and the license server. I was testing connectivity based on IP address and ports (seems reasonable — right?) All the nodes could reach the licensing server correctly. The issue was actually related to DNS failing and the indexing nodes could not resolve the license server address. I ensured the license server was entered into the hosts file and operations returned to normal.

View solution in original post

splunkoptimus
Path Finder

I had the same error I did all that @khourihan_splun  mentioned and tests were negative. to solve the problem I restarted splunkd on the indexer in question. 

0 Karma

nmurillo_splunk
Splunk Employee
Splunk Employee

Take a look at your pass4SymmKey in your server.conf.

0 Karma

khourihan_splun
Splunk Employee
Splunk Employee

We were seeing the SH return a message of DISABLED_DUE_TO_GRACE_PERIOD
This means the SH can't connect to the LM, which makes it seem like we have a DNS problem.

I experienced this error message in a Splunk 6.0.3 cluster.

Here's how I troubleshot it:

  • On the indexers, check the name of the slave-master listed in the server.conf
  • Check to see what IP address the slave master is resolving
  • Test connectivity from indexer to LM.

If you can not connect, you have a DNS issue, and as @matthewchase above described, the work around. But I'd recommend fixing your DNS ! 🙂

Another test is to run this on the licenser endpoint:

curl -k -u admin:changeme https://localhost:8089/services/licenser/localslave

mathewchase
Engager

So Splunk support never got back to me but I figured out the problem. The issue was indeed a communication problem between the indexing nodes and the license server. I was testing connectivity based on IP address and ports (seems reasonable — right?) All the nodes could reach the licensing server correctly. The issue was actually related to DNS failing and the indexing nodes could not resolve the license server address. I ensured the license server was entered into the hosts file and operations returned to normal.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...