Getting Data In

Filter events based on sourcetype and text at the same time

hernanrodriguez
Explorer

Hello guys

I have the following scenario:

  • I'm receiving a lot of logs from a Kubernetes Clusters
  • I'm sending logs from Kubernetes to a Splunk Heavy Forwarder using Splunk Connect for Kubernetes
  • The sourcetypes names are assigned by Splunk Connect using a structure like this: kube:container:* (example: kube:container:containerNumberOne)
  • I have the following confs in props.conf and transforms.conf files:
[(?::){0}kube:*]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
TRANSFORMS-set= setnull, allowEvents, dropEventsByText, dropEventsBySourcetype,  set_sourcetype

 

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[allowEvents]
REGEX = LOG_INI|LOG_FINOK|LOG_FINEX|LOG_FINNEG
DEST_KEY = queue
FORMAT = indexQueue

[dropEventsBySourcetype]
SOURCE_KEY=MetaData:Sourcetype
REGEX = containerNumberOne|containerNumberTwo
DEST_KEY = queue
FORMAT = nullQueue

[dropEventsByText]
REGEX = debug|DEBUG
DEST_KEY = queue
FORMAT = nullQueue

[set_sourcetype]
SOURCE_KEY=MetaData:Sourcetype
REGEX = kube\:container\:(.*)\-re\-
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

These filters (and the sourcetype rename) have been working well for a while and as you may observe, they filter events based on a text contained in the log or by a text in the sourcetypes name.

The problem is that I have a new requirement. I need to drop events based on 2 rules at the same time: a sourcetype name and a text in the log. Specifically, there are some logs with the sourcetype name containerFour and the text LOG_INI that I need to drop. I guess I need something like this (but I know te conf is wrong):

[dropEventsBySourcetypeAndText]
SOURCE_KEY=MetaData:Sourcetype
REGEX = containerNumberFour
REGEX = BCI_INI
DEST_KEY = queue
FORMAT = nullQueue

 

 

Does someone know what i need to do?

Thanks in advance 

 

Labels (3)
0 Karma
1 Solution

hernanrodriguez
Explorer

Finally I resolved my problem filtering by "source". The structure for containers is something like this: /var/log/containers/*log

So, my stanza now is:

[source::/var/log/containers/(containerNumberFour*|containerNumberFive*)]

Thanks @scelikok for your help

 

 

 
 
 
 
 

View solution in original post

0 Karma

hernanrodriguez
Explorer

Finally I resolved my problem filtering by "source". The structure for containers is something like this: /var/log/containers/*log

So, my stanza now is:

[source::/var/log/containers/(containerNumberFour*|containerNumberFive*)]

Thanks @scelikok for your help

 

 

 
 
 
 
 
0 Karma

mattymo
Splunk Employee
Splunk Employee

Glad you got it working!

You can also optimize by moving those filters to the collector. If you don't want the data, it can be filtered at the inputs just like a UF!! By default we pick up all the things...

 

https://github.com/splunk/splunk-connect-for-kubernetes#managing-sck-log-ingestion-by-using-annotati...

 

Best way to get rid of logs is to never pick them up!!!

- MattyMo
0 Karma

scelikok
SplunkTrust
SplunkTrust

You're welcome @hernanrodriguez,

Since it is regex below should work;

[(?::){0}kube:containerNumberFour.*]
TRANSFORMS-set= drop_BCI_INI_EventsByText

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

hernanrodriguez
Explorer

It didn't work 😞

Actually one of the sourcetypes is a bit different:

[(?::){0}kube:container:ms\-loan\-mobile\-exp.*]

Could be a problem with the special characters: "-"?

something like this should work?

[.*ms\-loan\-mobile\-exp.*]

0 Karma

scelikok
SplunkTrust
SplunkTrust

It seems, it may work. It is better to test your regex with www.regex101.com .

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @hernanrodriguez,

You can use separate props stanza using specific sourcetype and transforms for this requirement like below;

props.conf
[(?::){0}kube:containerNumberFour]
TRANSFORMS-set= drop_BCI_INI_EventsByText

transforms.conf
[drop_BCI_INI_EventsByText]
REGEX = BCI_INI
DEST_KEY = queue
FORMAT = nullQueue

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

hernanrodriguez
Explorer

I tried that yesterday, but It wasn't work.

The sourcetype is dynamic, so I configured something like this:

 

[(?::){0}kube:containerNumberFour*]
TRANSFORMS-set= drop_BCI_INI_EventsByText

 Can I use the * symbol to match any sourcetype with the prefix "containerNumberFour"?

I mean, my sourcetypes are something like:

kube:containerNumberFour-rev-1.1

kube:containerNumberFour-rev-1.2

kube:containerNumberFour-rev-2.0

 

Thanks @scelikok for your help

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...