topic Re: How can I debug my logs and whitelist a word? in Getting Data In

How can I debug my logs and whitelist a word?

guru865 — Tue, 29 Aug 2017 21:30:43 GMT

Hi Everyone.

How to discard all the debug logs for a sourcetype and whitelist a word "AuthIDDetection" whenever this comes in events from sourcetype

Please, could someone help with this --- I have sourcetype "xyz" to which I am discarding keyword "debug" from the events, we are discarding all the debug logs. I would like to discard all the debug logs for sourcetype -- xyz and whitelist a word "AuthIDDetection" whenever this comes in events from sourcetype "xyz" from the same debug logs.

current props.conf --

[sourcetype-xyz]
TRANSFORMS-set=xyz-setnull,setparsing

transforms.conf --

[xyz-setnull]
REGEX= debug|\\|Notice
DEST_KEY=queue
FORMAT=nullQueue

Could you please help with this.

FYI , Am following this documentation - http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Discard_specific_events_and_keep_the_res

Re: How can I debug my logs and whitelist a word?

jkat54 — Tue, 29 Aug 2017 23:18:29 GMT

Try this:

Props:
[sourcetype-xyz]
TRANSFORMS-AAA=xyz-whitelist
TRANSFORMS-ZZZ=xyz-setnull

Transforms:
[xyz-whitelist]
REGEX= AuthIDDetection
DEST_KEY=queue
FORMAT=indexQueue

[xyz-setnull]
REGEX= debug|\|Notice
DEST_KEY=queue
FORMAT=nullQueue

Re: How can I debug my logs and whitelist a word?

guru865 — Tue, 29 Sep 2020 15:34:06 GMT

Thanks @jkat54 for the inputs . Sure , i will check this as well . Also could you please suggest the below written transforms.conf in tandem with props.conf. Can i use setparsing stanza instead of white list in transforms.conf

Props :

[xyz_server]
TRANSFORMS-set= xyz-setnull,setparsing
pulldown_type = true
MAX_EVENTS = 20000

transforms.conf

[xyz-setnull]
REGEX = debug|Notice
DEST_KEY = queue
FORMAT = nullQueue

[xyz-setparsing]
REGEX= AuthIDDetection
DEST_KEY=queue
FORMAT=indexQueue
.

Re: How can I debug my logs and whitelist a word?

jkat54 — Wed, 30 Aug 2017 18:48:38 GMT

Your example would send everything to setnull first...

do this instead:
TRANSFORMS-set= xyz-setparse,xyz-setnull

My example had it reversed but I corrected it.

Re: How can I debug my logs and whitelist a word?

jkat54 — Thu, 31 Aug 2017 20:04:47 GMT

After the chat on slack, here's what you need for your regex and you only need one transforms.

.*\<Notice\>((?!AuthID).)*$

Re: How can I debug my logs and whitelist a word?

guru865 — Thu, 31 Aug 2017 22:13:37 GMT

Thanks alot @jkat54 . Yes this works . Setting this to nullQueue does the work

transforms.conf

 [xyz-setnull]
 REGEX =  .*\((?!AuthID).)*$
   DEST_KEY = queue
 FORMAT = nullQueue