Sending data to nullqueue using props and transafoms is not working.

[SagarSplunk]

Hi All,

I am trying to send data to nullqueue so that events will not get indexed. we can save license consumption.

Props.conf

[testfiltering]
DATETIME_CONFIG =
NO_BINARY_CHECK = true
category = Custom
disabled = false
pulldown_type = true
TRANSFORMS-SERVICE = eventsDrop

transforms.conf

[eventsDrop]
REGEX = (?m)^THREAD.SERVICE-.*E2ELoggingSupport.
DEST_KEY = queue
FORMAT = nullQueue

Log details to be filtered
2017-07-05 15:54:30.157 INFO THREAD-1321 SERVICE-[MDP Feeder]_BusinessFlowSelectorService_H075F54304221O1P H075F54304321O1Q E2ELoggingSupport : Payment Id: H075F54304321O1Q, JMS msg received header [Destination=queue:///GPP.FROMDP.SEND.PAYMNT.INSTRCTN.IN,DeliveryMode=2,Expiration=0 null,Priority=4,MessageID=ID:414d51204445564750503032202020205959ceef1000b103,Timestamp=1499233913142 2017-07-05T15:51:53.142,CorrelationID=null,ReplyTo=null,Redelivered=false,Type=null] PropertyNames=[JMS_IBM_Format=MQSTR ][JMS_IBM_Character_Set=UTF-8][JMSXDeliveryCount=1][JMS_IBM_Encoding=273][JMSXUserID=pegapsup ][JMS_IBM_MsgType=8][JMS_IBM_PutApplType=28][JMS_IBM_PutDate=20170705][JMS_IBM_PutTime=05512391][JMSXAppID=hermes.browser.HermesBrowser]

Thanks/Sagar

[s2_splunk]

Your RegEx anchors "THREAD" to the beginning of the line, but it doesn't show up at the beginning of the line. Either add the patterns for timestamp and category to your RegEx or remove the caret (^).

Also, make sure you put those configs where the parsing occurs; probably your indexers.

[SagarSplunk]

HI SSievert,
Now I changed my configurations as below but still I am unable to filter out the above events. am I missing something? syntax is correct for regex? I trying to filter out events before it index

[eventsDrop]
REGEX = SERVICE-.E2ELoggingSupport.
DEST_KEY = queue
FORMAT = nullQueue

Thanks/Sagar

[s2_splunk]

You need an "*" after the first "." to match on more than just one character. You can also skip the last "." Try this:
[eventsDrop]
REGEX = SERVICE-.*E2ELoggingSupport
DEST_KEY = queue
FORMAT = nullQueue
Drop it on your indexer and restart Splunk.

BTW, RegExr is a good tool to test whether your RegEx constructs work. 😉

[SagarSplunk]

Hi SSievert

I tried above Regex its too not working for me are there limitation for free version of splunk.

[s2_splunk]

There are limitations in the free version of Splunk, but this is not one of them.
If you make sure that

• the stanza name in props.conf matches your sourcetype and
• the stanza name in transforms matches what you used after TRANSFORMS-xxxx= and
• your RegEx works and matches what you want to match and
• you deploy props/transforms in the right place (where parsing happens, i.e. indexer or heavy forwarder, NOT universal forwarder)
• you restart splunk or reload the configuration after making the change

this will work with any version of Splunk Enterprise.