Splunk Search

how to not index some data or send it to null queue

surekhasplunk
Communicator

Hi,

I want to know if there is some mechanism by which i can stop indexing a particular kind of data like if
segment_name="Enforced segment"

From getting indexed.

My inputs.conf has following entry

[monitor:///opt/splunk/logs/check//.log]
disabled = 0
host_segment = 5
sourcetype = check_logs
index = check

here i dont want those lines to get indexed if any of the log files has this pattern in it (segment_name="Enforced segment")

Is it possible ?

Thanks

Tags (3)
0 Karma
1 Solution

manjunathmeti
SplunkTrust
SplunkTrust

Yes, add these configurations and check:

props.conf

[check_logs]
TRANSFORMS-null_queue = data_nullq

transforms.conf

[data_nullq]
DEST_KEY = queue
REGEX = segment_name=\"Enforced segment\"
FORMAT = nullQueue

View solution in original post

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

Yes, add these configurations and check:

props.conf

[check_logs]
TRANSFORMS-null_queue = data_nullq

transforms.conf

[data_nullq]
DEST_KEY = queue
REGEX = segment_name=\"Enforced segment\"
FORMAT = nullQueue
0 Karma

surekhasplunk
Communicator

Hi @manjunathmeti ,

thanks for quick reply

Only modification i did is i added like below for REGEX
REGEX = (segment_name=Enforced segment)

This will work right ? since i dont have that double quotes just equalto symbol is there.

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

yes, this will work.

0 Karma

surekhasplunk
Communicator

Thanks @manjunathmeti,

I have one more query if you are aware how to confirm that those have started going to the nullqueue?
where can i check to get an confirmation that they are now going to the null queue

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

Check: index=_internal sourcetype=splunkd component=metrics processor=nullqueue group=pipeline

0 Karma

surekhasplunk
Communicator

Thanks a lot ..
For now am not seeing anything related to my configuration change. but i think will something soon ..

0 Karma

surekhasplunk
Communicator

Hi @manjunathmeti ,

Now the issue is they are getting blocked but other indexes are also effected by this change dont know why

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

If you are using same sourcetype name for other indexes or monitors then this chnage will affect them. You can set unique sourcetype to this monitor or change stanza in propsc.conf as below:

[source::/opt/splunk/logs/check/*.log]
TRANSFORMS-null_queue = data_nullq
0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...