topic Re: Discarding log entries by specific content in Getting Data In

Discarding log entries by specific content

rijhwani — Wed, 13 Jun 2012 19:18:44 GMT

In /var/log/messages on numerous machines I have the following messages:

Jun 13 19:55:34 hostabc snmpd[27898]: Received SNMP packet(s) from UDP: [p.q.r.s]:46999 
Jun 13 19:55:34 hostabc snmpd[27898]: Connection from UDP: [p.q.r.s]:46999

I just want to eradicate these particular messages. I Ideally, I would like to not have them forwarded from to the indexers in the first place, but as a first goal I just want to stop them being indexed.

Taking my cues from Splunk online docs I have made config additions on both the indexers:

In etc/local/props.conf:

[source::/var/log/messages]
TRANSFORMS-null= ditchCacti

In etc/local/transforms.conf:

[ditchCacti]
REGEX = from UDP: \[p.q.r.s\]:
DEST_KEY = queue
FORMAT = nullQueue

Splunk has been restarted, but the offending messages continue to be indexed.

Am I on the right track, or have I completely misunderstood something?

Re: Discarding log entries by specific content

sdaniels — Wed, 13 Jun 2012 19:26:45 GMT

You are on the right track but the regex is not matching. The periods in p.q.r.s need to be escaped and account for the spaces. I think this should work for you.

REGEX = from\sUDP:\s\[p\.q\.r\.s\]:

Re: Discarding log entries by specific content

rijhwani — Wed, 13 Jun 2012 19:40:49 GMT

I can understand the spaces needing escaping, but whilst it is sloppy I would have expected '.' in the regex to match anything, including a literal ".".

It hasn't worked, BTW.

Re: Discarding log entries by specific content

sdaniels — Wed, 13 Jun 2012 19:51:02 GMT

That is true, yes, i'm particular about how I do my regex to avoid any issues though. Everything I see looks correct assuming these config files are on the indexer.

Re: Discarding log entries by specific content

sdaniels — Wed, 13 Jun 2012 20:00:52 GMT

You may try routing based on sourcetype. Whatever sourcetype you have configured in inputs.conf. Try that in props.conf instead of the source.

[mysourcetype]
TRANSFORMS-null= ditchCacti

Re: Discarding log entries by specific content

rijhwani — Fri, 15 Jun 2012 09:38:49 GMT

The configs are indeed on the indexers.

It is only specific messages I want to lose, so diverting an entire sourcetype to the black hole is not suitable. (As you might guess - only the Cacti polling. Any other unexpected SNMP I want to see.)