Getting Data In
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

Filter events based on sourcetype and text at the same time

hernanrodriguez
Explorer

Hello guys

I have the following scenario:

  • I'm receiving a lot of logs from a Kubernetes Clusters
  • I'm sending logs from Kubernetes to a Splunk Heavy Forwarder using Splunk Connect for Kubernetes
  • The sourcetypes names are assigned by Splunk Connect using a structure like this: kube:container:* (example: kube:container:containerNumberOne)
  • I have the following confs in props.conf and transforms.conf files:
[(?::){0}kube:*]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
TRANSFORMS-set= setnull, allowEvents, dropEventsByText, dropEventsBySourcetype,  set_sourcetype

 

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[allowEvents]
REGEX = LOG_INI|LOG_FINOK|LOG_FINEX|LOG_FINNEG
DEST_KEY = queue
FORMAT = indexQueue

[dropEventsBySourcetype]
SOURCE_KEY=MetaData:Sourcetype
REGEX = containerNumberOne|containerNumberTwo
DEST_KEY = queue
FORMAT = nullQueue

[dropEventsByText]
REGEX = debug|DEBUG
DEST_KEY = queue
FORMAT = nullQueue

[set_sourcetype]
SOURCE_KEY=MetaData:Sourcetype
REGEX = kube\:container\:(.*)\-re\-
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

These filters (and the sourcetype rename) have been working well for a while and as you may observe, they filter events based on a text contained in the log or by a text in the sourcetypes name.

The problem is that I have a new requirement. I need to drop events based on 2 rules at the same time: a sourcetype name and a text in the log. Specifically, there are some logs with the sourcetype name containerFour and the text LOG_INI that I need to drop. I guess I need something like this (but I know te conf is wrong):

[dropEventsBySourcetypeAndText]
SOURCE_KEY=MetaData:Sourcetype
REGEX = containerNumberFour
REGEX = BCI_INI
DEST_KEY = queue
FORMAT = nullQueue

 

 

Does someone know what i need to do?

Thanks in advance 

 

Labels (3)
0 Karma
1 Solution

hernanrodriguez
Explorer

Finally I resolved my problem filtering by "source". The structure for containers is something like this: /var/log/containers/*log

So, my stanza now is:

[source::/var/log/containers/(containerNumberFour*|containerNumberFive*)]

Thanks @scelikok for your help

 

 

 
 
 
 
 

View solution in original post

0 Karma

hernanrodriguez
Explorer

Finally I resolved my problem filtering by "source". The structure for containers is something like this: /var/log/containers/*log

So, my stanza now is:

[source::/var/log/containers/(containerNumberFour*|containerNumberFive*)]

Thanks @scelikok for your help

 

 

 
 
 
 
 

View solution in original post

0 Karma

mattymo
Splunk Employee
Splunk Employee

Glad you got it working!

You can also optimize by moving those filters to the collector. If you don't want the data, it can be filtered at the inputs just like a UF!! By default we pick up all the things...

 

https://github.com/splunk/splunk-connect-for-kubernetes#managing-sck-log-ingestion-by-using-annotati...

 

Best way to get rid of logs is to never pick them up!!!

0 Karma

scelikok
Champion

You're welcome @hernanrodriguez,

Since it is regex below should work;

[(?::){0}kube:containerNumberFour.*]
TRANSFORMS-set= drop_BCI_INI_EventsByText

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote is appreciated.
0 Karma

hernanrodriguez
Explorer

It didn't work 😞

Actually one of the sourcetypes is a bit different:

[(?::){0}kube:container:ms\-loan\-mobile\-exp.*]

Could be a problem with the special characters: "-"?

something like this should work?

[.*ms\-loan\-mobile\-exp.*]

0 Karma

scelikok
Champion

It seems, it may work. It is better to test your regex with www.regex101.com .

If this reply helps you an upvote is appreciated.
0 Karma

scelikok
Champion

Hi @hernanrodriguez,

You can use separate props stanza using specific sourcetype and transforms for this requirement like below;

props.conf
[(?::){0}kube:containerNumberFour]
TRANSFORMS-set= drop_BCI_INI_EventsByText

transforms.conf
[drop_BCI_INI_EventsByText]
REGEX = BCI_INI
DEST_KEY = queue
FORMAT = nullQueue

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote is appreciated.
0 Karma

hernanrodriguez
Explorer

I tried that yesterday, but It wasn't work.

The sourcetype is dynamic, so I configured something like this:

 

[(?::){0}kube:containerNumberFour*]
TRANSFORMS-set= drop_BCI_INI_EventsByText

 Can I use the * symbol to match any sourcetype with the prefix "containerNumberFour"?

I mean, my sourcetypes are something like:

kube:containerNumberFour-rev-1.1

kube:containerNumberFour-rev-1.2

kube:containerNumberFour-rev-2.0

 

Thanks @scelikok for your help

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!