Getting Data In
Highlighted

How do I find and remove strings in logs from the Forwarder?

New Member

Hello,

I'm trying to send some antivirus logs from the forwarder into Splunk.

The logs I'm sending have a tendency to spam, for example:

13/09/2018 16:06:53 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:49467  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:55 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:56 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:57 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:58 No usable rule found    Blocked 192.168.0.40:56694  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:59 No usable rule found    Blocked 192.168.0.40:56694  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM

I want to be able to filter out lines in the log that say "No usable rule found".

I've tried adding the following to props.conf which I've copied into [C:\Program Files\SplunkUniversalForwarder\etc\system\local] directory, here is the line I added to props.conf:

[source:\path\to\log\log.txt]
SEDCMD-strip-detail-msg=^.*(listening on the port|[Nn]o usable rule found)*$

I have also tried messing with transform.conf too, but to no avail.

Any ideas guys?

0 Karma
Highlighted

Re: How do I find and remove strings in logs from the Forwarder?

SplunkTrust
SplunkTrust

Universal forwarders cannot filter events. You must use a heavy forwarder or put your props.conf settings on your indexers.

The SEDCMD setting is not correct. It must be in the format s/<search regex>/<replacement string>/. Replacing the entire event string will not remove the event. To remove unwanted events, send them to nullQueue like so:

props.conf

[source:\path\to\log\log.txt] 
TRANSFORMS-set = setnull,setparsing

transforms.conf

[setnull]
REGEX = (listening on the port|[Nn]o usable rule found)
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
---
If this reply helps you, an upvote would be appreciated.
0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.