How can I debug my logs and whitelist a word?

[guru865]

Hi Everyone.

How to discard all the debug logs for a sourcetype and whitelist a word "AuthIDDetection" whenever this comes in events from sourcetype

Please, could someone help with this --- I have sourcetype "xyz" to which I am discarding keyword "debug" from the events, we are discarding all the debug logs. I would like to discard all the debug logs for sourcetype -- xyz and whitelist a word "AuthIDDetection" whenever this comes in events from sourcetype "xyz" from the same debug logs.

current props.conf --

[sourcetype-xyz]
TRANSFORMS-set=xyz-setnull,setparsing

transforms.conf --

[xyz-setnull]
REGEX= debug|\\|Notice
DEST_KEY=queue
FORMAT=nullQueue

Could you please help with this.

FYI , Am following this documentation - http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Discard_specific_events...

[jkat54 · Accepted solution]

Try this:

Props:
[sourcetype-xyz]
TRANSFORMS-AAA=xyz-whitelist
TRANSFORMS-ZZZ=xyz-setnull

Transforms:
[xyz-whitelist]
REGEX= AuthIDDetection
DEST_KEY=queue
FORMAT=indexQueue

[xyz-setnull]
REGEX= debug|\|Notice
DEST_KEY=queue
FORMAT=nullQueue

[guru865]

Thanks @jkat54 for the inputs . Sure , i will check this as well . Also could you please suggest the below written transforms.conf in tandem with props.conf. Can i use setparsing stanza instead of white list in transforms.conf

Props :

[xyz_server]
TRANSFORMS-set= xyz-setnull,setparsing
pulldown_type = true
MAX_EVENTS = 20000

transforms.conf

[xyz-setnull]
REGEX = debug|Notice
DEST_KEY = queue
FORMAT = nullQueue

[xyz-setparsing]
REGEX= AuthIDDetection
DEST_KEY=queue
FORMAT=indexQueue
.

[jkat54]

Your example would send everything to setnull first...

do this instead:
TRANSFORMS-set= xyz-setparse,xyz-setnull

My example had it reversed but I corrected it.

[jkat54]

After the chat on slack, here's what you need for your regex and you only need one transforms.

.*\<Notice\>((?!AuthID).)*$

[guru865]

Thanks alot @jkat54 . Yes this works . Setting this to nullQueue does the work

transforms.conf

 [xyz-setnull]
 REGEX =  .*\((?!AuthID).)*$
   DEST_KEY = queue
 FORMAT = nullQueue