Getting Data In

How do I find and remove strings in logs from the Forwarder?

brandonmcgrath1
New Member

Hello,

I'm trying to send some antivirus logs from the forwarder into Splunk.

The logs I'm sending have a tendency to spam, for example:

13/09/2018 16:06:53 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:49467  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:55 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:56 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:57 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:58 No usable rule found    Blocked 192.168.0.40:56694  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:59 No usable rule found    Blocked 192.168.0.40:56694  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM

I want to be able to filter out lines in the log that say "No usable rule found".

I've tried adding the following to props.conf which I've copied into [C:\Program Files\SplunkUniversalForwarder\etc\system\local] directory, here is the line I added to props.conf:

[source:\path\to\log\log.txt]
SEDCMD-strip-detail-msg=^.*(listening on the port|[Nn]o usable rule found)*$

I have also tried messing with transform.conf too, but to no avail.

Any ideas guys?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Universal forwarders cannot filter events. You must use a heavy forwarder or put your props.conf settings on your indexers.

The SEDCMD setting is not correct. It must be in the format s/<search regex>/<replacement string>/. Replacing the entire event string will not remove the event. To remove unwanted events, send them to nullQueue like so:

props.conf

[source:\path\to\log\log.txt] 
TRANSFORMS-set = setnull,setparsing

transforms.conf

[setnull]
REGEX = (listening on the port|[Nn]o usable rule found)
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...