Installation

How to stop Splunk search head from exceeding data limit allowed by license

mysplunkbase
Explorer

How do I to stop Splunk search head from exceeding data limit allowed by license.

The search head is Splunk App for windows infrastructure and is indexing information from  AD Server and Win 10 workstation.

 

 

mysplunkbase_0-1601132870069.png

 

Labels (4)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Strictly speaking, search heads do not index any data.  Indexing (and violating a license) is done by indexers, although a single-instance Splunk server fills both roles.

If you are exceeding your license then you have 2 options:

1) Increase your license;

2) Reduce the amount of data Splunk ingests

Most customers are likely to choose #2.

So, how do you reduce the amount of data ingested?  There are several ways and I won't go into specifics here, but feel free to ask specific questions.

  • Turn off unneeded inputs
  • Define blacklists for unneeded Windows events
  • Define whitelists for desired Windows events
  • Discard the "boilerplate" that is attached to every Windows event

What is the screen shot telling us?

---
If this reply helps you, Karma would be appreciated.

View solution in original post

96nick
Communicator

This one is entirely dependent on your environment.

You mention search head, and you seem to have a lot of events, so I'm going to assume you're in a distributed environment. There is no built-in way of stopping indexing when you hit your license, so you'll have to get creative if you want to achieve this.

First, know what happens when you go over your license. The sequence of events that happen after you go over depend on what type of license you have. Some more info is in the docs.

Secondly, one way you could stop indexing when you get to a certain % of your license used is to create an alert that runs a script that tells an app to stop collecting data. There was a previous Splunk Community post on that.

Thirdly, understand WHY you might go over and how you can fix it. Sometimes your friends in Russia decided to point a scanner on all of your servers and aggressively scan that night and you go over, that's not really something you can predict and fix. It's not always realistic to just get a bigger license, but if you truly need that data then maybe you can fine tune how often you receive it, or possibly turn off a data source that you aren't really using. I'd suggest taking a data inventory to figure out the whats and the whys of a particular data source in Splunk. Ask yourself the following questions:

  • What am I doing with this data? (alerts, reports, etc.)
  • Why am I collecting it? (use cases)
  • Do I need it every 30s? (changing that to 1min cuts your amount indexed in half!)
  • If I am collecting it, what can I do to turn that data into a powerful resource?

Hope this helped!

mysplunkbase
Explorer

Thank you!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Strictly speaking, search heads do not index any data.  Indexing (and violating a license) is done by indexers, although a single-instance Splunk server fills both roles.

If you are exceeding your license then you have 2 options:

1) Increase your license;

2) Reduce the amount of data Splunk ingests

Most customers are likely to choose #2.

So, how do you reduce the amount of data ingested?  There are several ways and I won't go into specifics here, but feel free to ask specific questions.

  • Turn off unneeded inputs
  • Define blacklists for unneeded Windows events
  • Define whitelists for desired Windows events
  • Discard the "boilerplate" that is attached to every Windows event

What is the screen shot telling us?

---
If this reply helps you, Karma would be appreciated.

mysplunkbase
Explorer

Thank you

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...