topic Re: Exclude some events from being indexed in Splunk Enterprise

Exclude some events from being indexed

pil321 — Thu, 03 Aug 2017 13:03:16 GMT

Calling all regex gurus!

I’m trying to drop all traffic with a certain IP (192.168.1.1) or a certain port number (123). This is what the log looks like:

2017-08-03 10:39:19,2017-08-03 10:39:19,0.000,192.168.1.1,192.168.6.225,123,123,,....

I found an answer for a way to do this (https://answers.splunk.com/answers/96/how-do-i-exclude-some-events-from-being-indexed-by-splunk.html).

This is what I have for my props.conf:

[source::/some/directory/in/splunk]
TRANSFORMS-set = null_1

This is transforms.conf:

[null_1]
REGEX = 192\.168.\1\.1
DEST_KEY = queue
FORMAT = nullQueue

I’m pretty sure the problem is with the regex, but I don’t have any regex skill whatsoever. Dropping all port 123 traffic would work as well.

Re: Exclude some events from being indexed

FritzWittwer_ol — Thu, 03 Aug 2017 13:33:36 GMT

your regexp has a typo, it should be:

REGEX = 192\\.168\\.1\\.1

regex010 ist one of the helpful online regular expressions checkers

Re: Exclude some events from being indexed

bheemireddi — Tue, 29 Sep 2020 15:12:12 GMT

I didn't test it myself, but a quick spot you missed a "\" in before one of the dots. may be you can try this one. I added "," as well to make sure it is getting from the right place.

[null_1]
REGEX = ,192\.168\.1\.1,
DEST_KEY = queue
FORMAT = nullQueue

Re: Exclude some events from being indexed

pil321 — Thu, 03 Aug 2017 13:43:01 GMT

Yep...I went there. The thing is...my expression also works on that site. The \. is meant to literally match the .

In your case the . is matching everything after the numbers....so your expression works as well.

I can give your expression a try and see.

Re: Exclude some events from being indexed

pil321 — Thu, 03 Aug 2017 13:49:08 GMT

Sorry folks....the typo was on the code in the post...not on the actual configs!

This is what I have in the configs: 192\.168\.1\.1

Re: Exclude some events from being indexed

FritzWittwer_ol — Thu, 03 Aug 2017 14:09:26 GMT

I'd try

REGEX = .\*192\\.168\\.1\\.1.\*

but the .* should not be needed, so eventually a config wich is not seen or overriden, did you try btool to verify the configuration?

Re: Exclude some events from being indexed

sbbadri — Thu, 03 Aug 2017 14:16:14 GMT

try this

[source::/some/directory/in/splunk]
TRANSFORMS-null = setnull

[setnull]
REGEX = \,192\.168\.1\.1\,

DEST_KEY = queue

FORMAT = nullQueue

---- or -----

[setnull]
REGEX =\d+-\d+-d+\s\d+:\d+:\d+\,\d+-\d+-d+\s\d+:\d+:\d+\, \,192\.168\.1\.1\,

DEST_KEY = queue

FORMAT = nullQueue

Re: Exclude some events from being indexed

bheemireddi — Tue, 29 Sep 2020 15:12:17 GMT

pil321,
anything with .* may be matching with lot more stuff than you think, to be precise if you just want to match with an IP address field, I wouldn't use .*
I just did a quick test and below regex should solve yours, if all you are looking to drop the events with that IP match 192\.168\.1\.1

[null_1]
REGEX = ,192\.168\.1\.1,
DEST_KEY = queue
FORMAT = nullQueue

Re: Exclude some events from being indexed

alemarzu — Thu, 03 Aug 2017 14:31:39 GMT

This was configured on the indexer ?

Re: Exclude some events from being indexed

gcusello — Thu, 03 Aug 2017 15:02:25 GMT

Hi pil321,
I usually use sourcetype in filters to be sure that it runs!
So try something like this:

in props.conf

[your_sourcetype]
TRANSFORMS-set-exclude=set_exclude,set_nullqueue

in transforms.conf

[set_exclude]
REGEX=.
DEST_KEY = queue
FORMAT = indexQueue
[set_nullqueue]
REGEX=(192\.168\.1\.1)|(,123,)
DEST_KEY=queue
FORMAT=nullQueue

Bye.
Giuseppe

Re: Exclude some events from being indexed

gcusello — Thu, 03 Aug 2017 15:02:25 GMT

Hi pil321,
I usually use sourcetype in filters to be sure that it runs!
So try something like this:

in props.conf

[your_sourcetype]
TRANSFORMS-set-exclude=set_exclude,set_nullqueue

in transforms.conf

[set_exclude]
REGEX=.
DEST_KEY = queue
FORMAT = indexQueue
[set_nullqueue]
REGEX=(192\.168\.1\.1)|(,123,)
DEST_KEY=queue
FORMAT=nullQueue

Bye.
Giuseppe

Re: Exclude some events from being indexed

SteveSmi — Sat, 21 Feb 2026 20:59:21 GMT

the issue is definitely your regex.

This part is wrong:

192\.168.\1\.1

\1 doesn’t make sense there unless you previously captured something. You just want to match the literal IP. To drop all events containing 192.168.1.1, your transforms.conf should look like this:

[null_1] REGEX = 192\.168\.1\.1 DEST_KEY = queue FORMAT = nullQueue

That will drop any event that contains that IP anywhere in the line.

If instead you want to drop all traffic with port 123, and assuming the port appears as a full field like in your example:

...,192.168.1.1,192.168.6.225,123,123,...

You can match it like this:

REGEX = ,123,

That makes sure you’re matching the port field and not something random like part of another number.

If you want to drop events that contain either that IP or port 123, you can combine them:

REGEX = 192\.168\.1\.1|,123,

That’s all you need.