Getting Data In

Best way to find log patterns in splunk consuming more bandwidth

Hi All,

I am looking for the best way to find log patterns in splunk consuming more bandwidth so that we can reduce the noise from splunk and control the license utilization.

0 Karma

Legend

Hi manishsingh777,
Could you share more details of your request?
What do you mean with:

  • "noise"?
  • "log patterns in splunk consuming more bandwidth"?

The only way to reduce license utilization is to analyze your flows (one by one) finding the regexes of the logs to discard, so you can apply these filters to Indexers.
Obvioulsy, if you discard logs, you cannot use them!

Sorry for my little answer but it's not possible to do more without information!

Bye.
Giuseppe

0 Karma

@gcusello

I am trying to find the log patterns in splunk which are consuming more license, the entire agenda behind this exercise to find those logs and then fine tune them to reduce the license consumption. Let me know if you need any more information.

0 Karma

Legend

Hi manishsingh777,
to understand the more consuming patterns, you have to run an analysis starting from License consuption by sourcetype dashboard, you can find it at [Settings -- Licensing -- Usage Report -- Previous 30 Days -- Split by sourcetype], eventually I suggest to open this panel in Search so you can see the sourcetype more expensive by numeric value.
Then you can run a search for that sourcetype and identify some fields (e.g. for wineventlog EventCode, for some firewall the LogLevel, etc...), than you have to know the meaning of the values, e.g. for wineventlog there are some EventCodes that could be not interesting for you, e.g. you could run something like this

index=wineventlog sourcetype=wineventlog:Security 
| stats values(EventCodeDescription) AS EventCodeDescription count By EventCode
| sort -count

so you can identify the EventCodes with more events and analyze if you need them or not.

When you identified the values to use to filter events, you can build the regex to use in filter, e.g.:

EventId\s+\=\s+(1234|1235|1236)

So you can filter your events as described in https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Filter_event_data... .

Bye.
Giuseppe

0 Karma