Installation

How to stop Splunk search head from exceeding data limit allowed by license

mysplunkbase
Explorer

How do I to stop Splunk search head from exceeding data limit allowed by license.

The search head is Splunk App for windows infrastructure and is indexing information from  AD Server and Win 10 workstation.

 

 

mysplunkbase_0-1601132870069.png

 

Labels (4)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Strictly speaking, search heads do not index any data.  Indexing (and violating a license) is done by indexers, although a single-instance Splunk server fills both roles.

If you are exceeding your license then you have 2 options:

1) Increase your license;

2) Reduce the amount of data Splunk ingests

Most customers are likely to choose #2.

So, how do you reduce the amount of data ingested?  There are several ways and I won't go into specifics here, but feel free to ask specific questions.

  • Turn off unneeded inputs
  • Define blacklists for unneeded Windows events
  • Define whitelists for desired Windows events
  • Discard the "boilerplate" that is attached to every Windows event

What is the screen shot telling us?

---
If this reply helps you, Karma would be appreciated.

View solution in original post

96nick
Communicator

This one is entirely dependent on your environment.

You mention search head, and you seem to have a lot of events, so I'm going to assume you're in a distributed environment. There is no built-in way of stopping indexing when you hit your license, so you'll have to get creative if you want to achieve this.

First, know what happens when you go over your license. The sequence of events that happen after you go over depend on what type of license you have. Some more info is in the docs.

Secondly, one way you could stop indexing when you get to a certain % of your license used is to create an alert that runs a script that tells an app to stop collecting data. There was a previous Splunk Community post on that.

Thirdly, understand WHY you might go over and how you can fix it. Sometimes your friends in Russia decided to point a scanner on all of your servers and aggressively scan that night and you go over, that's not really something you can predict and fix. It's not always realistic to just get a bigger license, but if you truly need that data then maybe you can fine tune how often you receive it, or possibly turn off a data source that you aren't really using. I'd suggest taking a data inventory to figure out the whats and the whys of a particular data source in Splunk. Ask yourself the following questions:

  • What am I doing with this data? (alerts, reports, etc.)
  • Why am I collecting it? (use cases)
  • Do I need it every 30s? (changing that to 1min cuts your amount indexed in half!)
  • If I am collecting it, what can I do to turn that data into a powerful resource?

Hope this helped!

mysplunkbase
Explorer

Thank you!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Strictly speaking, search heads do not index any data.  Indexing (and violating a license) is done by indexers, although a single-instance Splunk server fills both roles.

If you are exceeding your license then you have 2 options:

1) Increase your license;

2) Reduce the amount of data Splunk ingests

Most customers are likely to choose #2.

So, how do you reduce the amount of data ingested?  There are several ways and I won't go into specifics here, but feel free to ask specific questions.

  • Turn off unneeded inputs
  • Define blacklists for unneeded Windows events
  • Define whitelists for desired Windows events
  • Discard the "boilerplate" that is attached to every Windows event

What is the screen shot telling us?

---
If this reply helps you, Karma would be appreciated.

mysplunkbase
Explorer

Thank you

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...