topic Re: Filtering out data (from a forwarder) on Indexer? in Getting Data In

Filtering out data (from a forwarder) on Indexer?

spunk311z — Thu, 13 Feb 2020 16:36:32 GMT

hi, i have several universal forwarders deployed, and im getting lots of events i want to filter out.

I understand from reading answers here i need to do this on the indexer (or else install heavy forwaders on my endpoints, which i dont want to do).
This is a raw entry that im trying to drop / filter out from my indexer (ie to keep it from using up lots of my license):

02/13/2020 10:19:09.016
event_status="(0)The operation completed successfully."
pid=1216
process_image="c:\Program Files\VMware\VMware Tools\vmtoolsd.exe"
registry_type="CreateKey"
key_path="HKLM\system\controlset001\services\tcpip\parameters"
data_type="REG_NONE"
data=""

This is the entry from the inputs.conf on the forwarders that is sending some of the events i want to filter out:

    [WinRegMon://default]
    disabled = 0
    hive = .*
    proc = .*
    type = rename|set|delete|create

And i have added these lines on my indexer (and restarted), but im still seeing the events come in:

#on props.conf ( located in: C:\Program #Files\Splunk\etc\users\admin\search\local\props.conf):

[WinRegMon://default]
TRANSFORMS-set= setnull

#on transforms.conf ( located in: C:\Program #Files\Splunk\etc\users\admin\search\local\transforms.conf):

[setnull]
REGEX = process_image=.+vmtoolsd.exe"
DEST_KEY = queue
FORMAT = nullQueue

Thanks!
(ive been referencing many answers, including this good one):
(h)ttps:// answers.splunk.com/answers/37423/how-to-configure-a-forwarder-to-filter-and-send-the-specific-events-i-want.html

Re: Filtering out data (from a forwarder) on Indexer?

gcusello — Thu, 13 Feb 2020 16:56:56 GMT

Hi @spunk311z,
check the regex: = is a special char and must be escaped, try this

process_image\=\".*vmtoolsd\.exe\"

Ciao.
Giuseppe

Re: Filtering out data (from a forwarder) on Indexer?

spunk311z — Thu, 13 Feb 2020 22:39:25 GMT

Thanks, But the events are still coming in even with this in my transforms.conf:

[setnull]
REGEX = process_image\=\".*vmtoolsd\.exe\"
DEST_KEY = queue
FORMAT = nullQueue

(i had run my original regex through regex tester.com and got a full match, fwiw. same on yours too). tks
Have tried both reloading and restarting splunk server.

Re: Filtering out data (from a forwarder) on Indexer?

spunk311z — Thu, 13 Feb 2020 23:28:12 GMT

I was able to fix this, the issue does not appear to be with my orginal regex (although Giuseppe's suggestion is better form than mine and is what i will be using),

it was that i was editing props.conf and transform.conf in:
$SPLUNK_BASE$\etc\users\admin\search\local\

as opposed to the proper location of:
$SPLUNK_BASE$\etc\system\local\

once i copied my stanzas over to that location, and refreshed, the entries in question stopped coming in!

Can anyone explain the difference between .conf files in these two locations, please?

(i am logging into splunk with splunk user: admin).
thanks!

Re: Filtering out data (from a forwarder) on Indexer?

gcusello — Fri, 14 Feb 2020 07:58:26 GMT

Hi @spunk311z,
are you sure that "WinRegMon://default" is the sourcetype of the events to delete?
see them in Splunk.

It's correct to install Universal Forwarders on Endpoints, but have you any intermediate Heavy Forwarder between UFs and Indexers?

Ciao.
Giuseppe