topic Using Heavy Forwarded to Send Subset of Data to 3rd Party and Not Index in Getting Data In

Using Heavy Forwarded to Send Subset of Data to 3rd Party and Not Index

kmattox — Wed, 30 Sep 2020 00:31:13 GMT

Having issues with routing data to a 3rd party and then dropping the events from being indexed. The Windows event is being sent to the 3rd party but also is being indexed. I currently have a case open with support but wanted to ask the question to see if anyone has dealt with the issue before.

-bash-4.2$ more props.conf
[source::WinEventLog:Security]
TRANSFORMS-pta = pta_syslog_filter

[WinEventLog:Security]
TRANSFORMS-eventcodes = badevents

-bash-4.2$ more transforms.conf

Send eventcode 4624 to 3rd party

[pta_syslog_filter]
REGEX = .EventCode=4624.
DEST_KEY =_SYSLOG_ROUTING
FORMAT = pta_syslog

Windows events to drop. If I add 4624 below, the events are not sent to 3rd party.
[badevents]
REGEX=(?m)EventCode=(4634|560|562|5156|4689|4648|4662|4769|5061|5058)
DEST_KEY=queue
FORMAT=nullQueue

-bash-4.2$ more outputs.conf
[tcpout]
defaultGroup = default-autolb-group

[syslog:pta_syslog]
server = 3rdPartyHostIP:11514
sendCookedData = false
type=udp
timestampformat = %s

Splunk indexers
[tcpout:default-autolb-group]
server = indexer1:9997, indexer2:9997
autoLB = true

Re: Using Heavy Forwarded to Send Subset of Data to 3rd Party and Not Index

koshyk — Wed, 15 May 2019 20:38:40 GMT

You are very close. All You need is just to put the transforms in sequence and then you can add 4624 also to your badevents

[WinEventLog:Security]
TRANSFORMS-pta = pta_syslog_filter, badevents

Please note; I'ven't verified your REGEX is correct or NOT. I was just saying the approach only.

Re: Using Heavy Forwarded to Send Subset of Data to 3rd Party and Not Index

kmattox — Wed, 30 Sep 2020 00:31:47 GMT

Thanks for the suggestion but unfortunately I am getting the same result after making the changes. EventCode 4624 is still being indexed. Any other thoughts of what it could be?

-bash-4.2$ more props.conf
[source::WinEventLog:Security]
TRANSFORMS-pta = pta_syslog_filter, badevents

-bash-4.2$ more transforms.conf
[pta_syslog_filter]
REGEX = .EventCode=4624.
DEST_KEY =_SYSLOG_ROUTING
FORMAT = pta_syslog

[badevents]
REGEX=(?m)EventCode=(4634|560|562|5156|4689|4648|4662|4769|5061|5058)
DEST_KEY=queue
FORMAT=nullQueue

Re: Using Heavy Forwarded to Send Subset of Data to 3rd Party and Not Index

koshyk — Fri, 17 May 2019 12:43:44 GMT

that's coz you haven't put 4624 in your REGEX
Try..

[badevents]
REGEX=(?m)EventCode=(4624|4634|560|562|5156|4689|4648|4662|4769|5061|5058)
DEST_KEY=queue
FORMAT=nullQueue

Re: Using Heavy Forwarded to Send Subset of Data to 3rd Party and Not Index

kmattox — Fri, 17 May 2019 14:35:21 GMT

When I add 4624 to my REGEX, the events are not sent to the 3rd party. Is there a way to send 4624 events on the HF and then drop the events when they get to the indexers?

Re: Using Heavy Forwarded to Send Subset of Data to 3rd Party and Not Index

koshyk — Sat, 18 May 2019 10:37:05 GMT

if you put the props.conf stanza with pta_syslog_filter first, surely it will send to 3rd party and then ONLY action the badevents. You may need to playaround with

[source::WinEventLog:Security]
TRANSFORMS-pta = pta_syslog_filter, badevents

transforms.conf

[pta_syslog_filter]
REGEX = .*EventCode=4624.*
DEST_KEY =_SYSLOG_ROUTING
FORMAT = pta_syslog

[badevents]
REGEX=(?m)EventCode=(4624|4634|560|562|5156|4689|4648|4662|4769|5061|5058)
DEST_KEY=queue
FORMAT=nullQueue

Also you need to have stanza in outputs.conf

[syslog: pta_syslog]
server = loghost.example.com:514

to send it