We started getting license warnings on the license master and after updating the license to 15G I can confirm the ui shows 15gb, however gives a warning for the indexer connecting to it has run out of license. On the indexer when I list all the licenser messages I see:
d459b5b73aa7e48c37412f233c8f2237
category:license_window
create_time:1433203200
description:This pool has exceeded its configured poolsize=15 bytes. A warning has been recorded for all members
pool_id:auto_generated_pool_enterprise
severity:WARN
slave_id:D746D644-6CC5-4F2D-B6FF-A71EDB69607A
stack_id:enterprise
Somehow the indexer seems to think it is a 15byte license.
On the license master I see:
Licenses Volume Expiration Status
Splunk Enterprise 15,360 MB xxxxx valid Delete
Effective daily volume 15,360 MB
Pools Indexers Volume used today
auto_generated_pool_enterprise 835 MB / 15,360 MB Edit | Delete
<hostname> 835 MB (5.436%)
Add pool
After adding the license I ended up restarting license master as well as indexer with the hope that maybe the license was not being recognized. Now when I try to search an index I do not see any data at all. Do you think this could be because of the license warnings? Or does bouncing splunk cause data to be lost?
Definitely NOT data loss from bouncing Splunk.
If you have 5 license warnings in the past 30 days what Splunk does is it locks your searching capabilities. It won't allow any of your searches / dashboards to get data, but it will continue to index data so that when you resolve the problem, all the data that was sent during the license violation period, will be available.
If you have increased the license (which it seems like you have), you need to ask your sale rep to send you a reset key to clean out the license violation warnings. Once that's done your search capabilities will become available again.
Hope this helps
Thanks for your answer. Where is this reset key specified on Splunk?
Just like @acharlieh said, just upload it to your license master as if the reset key was a regular license you were uploading.
Once you obtain a reset license from your sales rep, you install it onto your license master. See the docs on License Violations for more information on this.