topic Re: How do you route windows event logs based on filters on a heavy forwarder? in Splunk Search

How do you route windows event logs based on filters on a heavy forwarder?

shivarpith — Tue, 29 Sep 2020 21:47:46 GMT

Hi,

We have had this working in the past, but for some reason, now, i am unable to forward filtered events to one Tcp group.

Both the output groups receive all events.

To summarize, the UF to send winevent:security to tcpout:hf group, HF to route winevent security logs based on the eventcode filters to indexers and syslog archivers.

but i am receiving all event codes to indexers as well as syslog.

On UF:
Inputs:
[WinEventLog://Security]
_TCP_ROUTING=HF
disabled = false
index=idx1
start_from = oldest
current_only = 1
evt_resolve_ad_obj = 1
checkpointInterval = 5

[WinEventLog://Application]
disabled = 0
index = idx1
start_from = oldest
current_only = 0
checkpointInterval = 5

Outputs:

[tcpout]
defaultGroup = indexers
forceTimebasedAutoLB=true
useACK = true
autoLBFrequency=15

[tcpout:indexers]
server = idx1:9997, idx2:9997, idx3:9997, idx4:9997, idx5:9997, idx6:9997

[tcpout:HF]
autoLB=true
server = HF1:9997,HF2:9997

HF Outputs

[tcpout]

defaultGroup = indexers

forceTimebasedAutoLB=true
useACK = true
autoLBFrequency=15
forwardedindex.2.whitelist = (_audit|_introspection|_telemetry|_internal)

[indexAndForward]
index=false

[tcpout:indexers]
server = idx1:9997, idx2:9997, idx3:9997, idx4:9997, idx5:9997, idx6:9997

[syslog] ## syslog outputs for archive##

[syslog:syslogGroup]
disabled=false
server=syslog:514

HF Transforms

[WinSecEvent-Splunk-SubSet]
REGEX = (?m)^EventCode=(4634|4776) # need to forward selected events to indexers

REGEX = (?m)^(.EventCode=(4634|4776).)

DEST_KEY=_TCP_ROUTING
FORMAT=indexers

[WinSecEvent-Syslog-All]
REGEX = (.) # Archive all event codes on a syslog archive
DEST_KEY=_SYSLOG_ROUTING
FORMAT=syslogGroup

What am i missing 😕

Thanks in Advance!!

Re: How do you route windows event logs based on filters on a heavy forwarder?

FrankVl — Thu, 25 Oct 2018 07:27:21 GMT

What does your HF props.conf look like?

Re: How do you route windows event logs based on filters on a heavy forwarder?

shivarpith — Thu, 25 Oct 2018 11:09:19 GMT

Looks like i missed sending that.

HF Props
[WinEventLog:Security]
TRANSFORMS-routing=WinSecEvent-Syslog-All,WinSecEvent-Splunk-SubSet
SEDCMD = s/[\t\n\r]/ /g

Re: How do you route windows event logs based on filters on a heavy forwarder?

FrankVl — Thu, 25 Oct 2018 11:28:43 GMT

Are you sure that sourcetype is correct? I know the TA for Windows assigns sourcetypes like that based on transforms, but that is too late for this props.conf to work. Not 100% sure anymore what the sourcetype will be before that transforms applies...

Does that SEDCMD work properly for you?

Re: How do you route windows event logs based on filters on a heavy forwarder?

harsmarvania57 — Thu, 25 Oct 2018 11:42:56 GMT

Can you please check $SPLUNK_HOME/var/log/splunk/metrics.log on Indexers and check whether you are receiving WinEventLog:Security source OR sourcetype from UF ?

Re: How do you route windows event logs based on filters on a heavy forwarder?

shivarpith — Tue, 29 Sep 2020 21:45:30 GMT

I am receiving winevent:security logs from these machines to splunk,

my concern is that, i am unable to filter based on the event codes as suggested in splunk docs and answers. i am getting all event codes to splunk which i do not want. i only want a specific set of event codes that i can add in HF transforms.

INFO Metrics - group=per_sourcetype_thruput, series="wineventlog:security", kbps=0.06895835463992131, eps=0.06451654193812648, kb=2.1376953125, ev=2, avg_age=2, max_age=2

Re: How do you route windows event logs based on filters on a heavy forwarder?

shivarpith — Fri, 26 Oct 2018 14:41:25 GMT

SEDCMD is working as expected.

HF is only acting as a relay so it should carry the sourcetype to further output groups.
it for some reason is either unable to apply the transforms or is unable to regex defined in the transforms.