Getting Data In

What is the best way to find log patterns in Splunk consuming more bandwidth?

manish_singh_77
Builder

Hi All,

I am looking for the best way to find log patterns in splunk consuming more bandwidth so that we can reduce the noise from splunk and control the license utilization.

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi manish_singh_777,
Could you share more details of your request?
What do you mean with:

  • "noise"?
  • "log patterns in splunk consuming more bandwidth"?

The only way to reduce license utilization is to analyze your flows (one by one) finding the regexes of the logs to discard, so you can apply these filters to Indexers.
Obvioulsy, if you discard logs, you cannot use them!

Sorry for my little answer but it's not possible to do more without information!

Bye.
Giuseppe

0 Karma

manish_singh_77
Builder

@gcusello

I am trying to find the log patterns in splunk which are consuming more license, the entire agenda behind this exercise to find those logs and then fine tune them to reduce the license consumption. Let me know if you need any more information.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi manish_singh_777,
to understand the more consuming patterns, you have to run an analysis starting from License consuption by sourcetype dashboard, you can find it at [Settings -- Licensing -- Usage Report -- Previous 30 Days -- Split by sourcetype], eventually I suggest to open this panel in Search so you can see the sourcetype more expensive by numeric value.
Then you can run a search for that sourcetype and identify some fields (e.g. for wineventlog EventCode, for some firewall the LogLevel, etc...), than you have to know the meaning of the values, e.g. for wineventlog there are some EventCodes that could be not interesting for you, e.g. you could run something like this

index=wineventlog sourcetype=wineventlog:Security 
| stats values(EventCodeDescription) AS EventCodeDescription count By EventCode
| sort -count

so you can identify the EventCodes with more events and analyze if you need them or not.

When you identified the values to use to filter events, you can build the regex to use in filter, e.g.:

EventId\s+\=\s+(1234|1235|1236)

So you can filter your events as described in https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Filter_event_data... .

Bye.
Giuseppe

0 Karma

Taruchit
Contributor

Hi @gcusello,

I think I have a similar use case as @manish_singh_77. Following are the details: -

I have wineventlogs and more than 50% of them are of one EventCode; thus, it is a noisy event code.

I need your help to understand how to analyze the data and find a pattern that is causing the noise. 

The goal is to determine what can be filtered out without losing the visibility. 

Thank you

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Taruchit,

at first it isn't a good idea to add another question, even if on the same topic, to another question because less people will read and answer it.

Anyway, you have to add to your inputs.conf, in the wineventlog stanza a blacklist statement, as you can read at https://docs.splunk.com/Documentation/Splunk/9.0.1/Admin/Inputsconf in the Event Log Filtering section.

In other words, you have to add to your inputs.conf stanza:

 

blacklist1 = <your_EventCode>

 

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...