Security

Getting a pool warning for a pool that has 100 meg allocated in a 2 gig license but LM shows only 23 MB indexed today!

wrangler2x
Motivator

We are sharing 100 megabytes of our 2 gigabytes daily license with another system that monitors an Apache web log. The volume of log data is typically around 25 MB a day. URL/manager/system/licensing page shows only 23 MB used by this pool so far today, but the warning we are getting on the yellow warning bar is:

Daily indexing volume limit exceeded for 1 slaves. See License Manager for details.

Clicking on that the message is:

2 pool warnings reported by 1 indexer   Correct by midnight to avoid violation

Drilling down, we see

indexing quota exceeded for this pool, poolsz=104857600 bytes

Why is that being generated? Had this four times last week. At midnight this will make a fifth violation. Previously the log file was many gigabytes, and based on splunkd.log was getting re-read, so those four I understand. But yesterday I rolled-out that log, and hup'd the Apache server creating the logs, so started with new log file at 0 bytes, now ~23 megs. followtail = 0 in inputs.conf. Any ideas what the heck is going on?

1 Solution

wrangler2x
Motivator

I think I get what this is now. The 5 warnings must be the four warnings
we saw yesterday, plus the one generated at midnight last night. So I think
it is announcing not that this pool went over quota today, but that it has
five times in the past.

And the second message must be a confirmation that these 5 warnings
places the pool in violation of the license.

Can anyone confirm if this is what is going on?

View solution in original post

0 Karma

wrangler2x
Motivator

I think I get what this is now. The 5 warnings must be the four warnings
we saw yesterday, plus the one generated at midnight last night. So I think
it is announcing not that this pool went over quota today, but that it has
five times in the past.

And the second message must be a confirmation that these 5 warnings
places the pool in violation of the license.

Can anyone confirm if this is what is going on?

0 Karma

wrangler2x
Motivator

That is in fact what was going on.

0 Karma

gryz
Explorer

Did you switch to a local Master License server at some point?

I had something similar due to switching to a local local master and then back.

To fix it, I had to switch back to local Master , get a reset key and then switched back to being a slave.

Seems similar somehow ...

0 Karma
Get Updates on the Splunk Community!

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...