topic Re: Windows Event Forwarding custom channels, renaming sources, adding metadata in Getting Data In

Windows Event Forwarding custom channels, renaming sources, adding metadata

jcapmany — Thu, 21 Feb 2019 09:28:57 GMT

We have a custom Windows Event Forwarding deployment, with specific channels (i.e. not all goes to ForwardedEvents).
FWD/Application, FWD/System, and so on.

We use Splunkforwarder (7.2.1) to get this all into splunk. It reads the sources just fine, with source::WinEventLog:FWD-name of channel.

Windows TA (5.0) is not very fond of this, as it searches for source::WinEventLog:Security or a known, standard channel name.

No matter, override [source::WinEventLog:FWD-Security] for example, and apply there the transforms (custom app to override this goes in forwarder, indexers and search head). This works just fine. All transforms are applied.

Since we like to have the host as the source machine, we add a meta field, wec_host, for troubleshooting purposes.

[set-wef-forwarding-host]
REGEX = (?m)host::(.*)$
SOURCE_KEY = MetaData:Host
FORMAT = wec_host::$1
WRITE_META = true

Works like a charm. Then, we want to apply another transform to add the original channel it was received from to the metadata, which is present in the source:

[set-wef-channel]
REGEX = WinEventLog:(.*)
SOURCE_KEY = MetaData:Source
FORMAT = wef_channel::$1
WRITE_META = true

After that, we apply the usual transforms from Windows TA to fix the source and sourcetype:

[source::WinEventLog:FWD-Security]
TRANSFORMS-t1-add_forwarder_for_wef = set-wef-forwarding-host
**TRANSFORMS-t2-add_channel_for_wef = set-wef-channel**
TRANSFORMS-t3-change_xml_host_for_windows_wef = WinEventXmlHostOverride
TRANSFORMS-t4-fix_source_and_sourcetype = ta-windows-fix-xml-source,ta-windows-fix-sourcetype

Taking into account top to bottom, and precedence, checking with btool, it appears the transforms should be applying in the right order, but by the time the set-wef-channel transform hits, the MetaData:Source no longer contains WinEventLog:FWD-Security and is empty as the regex doesn't match.

What are we missing?

Thanks in advance.

Re: Windows Event Forwarding custom channels, renaming sources, adding metadata

amielke — Mon, 13 May 2019 18:57:36 GMT

We use

[source:WinEventLog:Security]
TRANSFORMS-classname = Transforms_stanza

and it works for Security fine.

We have the problem at Windows Event Collector:

[source:WinEventLog:WEC/Channel1]
TRANSFORMS-classname = ....

We try

source::../Channel1
or
source::...//Channel1 OR source::..WEC/Channel1 OR source::...\Channel1 OR  source::...Channel1

but no solution! 😞

Some Ideas?

Re: Windows Event Forwarding custom channels, renaming sources, adding metadata

jcapmany — Tue, 14 May 2019 08:00:00 GMT

Use a dash instead of a backslash:
e.g. [source::WinEventLog:WEC-Channel1] instead of [source:WinEventLog:WEC/Channel1]

Re: Windows Event Forwarding custom channels, renaming sources, adding metadata

amielke — Wed, 15 May 2019 07:58:06 GMT

The dash doesn't help 😞

Re: Windows Event Forwarding custom channels, renaming sources, adding metadata

jcapmany — Wed, 15 May 2019 13:28:41 GMT

Yikes, sorry, I fat fingered that one:

This should work in your inputs.conf:
[WinEventLog://WEC-Security]

Then you would refer to it in props.conf as:
[source::WinEventLog:WEC-Security]

At this stage you can manipulate the source and sourcetype with the standard windows TA transforms if you so choose.

Re: Windows Event Forwarding custom channels, renaming sources, adding metadata

Lombi — Mon, 31 Aug 2020 09:29:01 GMT

Hey guys,

I just stumbled over the same issues. Besides remembering the WEC Host name I wanted to have a field with the original (custom) Channel Name. After some playing around I got this config on my indexers and it seems to work fine for me.

props.conf

[WinEventLog:ForwardedEvents] TRANSFORMS-change_host_for_windows_wef = WinEventRememberHost, WinEventHostOverride [XmlWinEventLog:ForwardedEvents] TRANSFORMS-change_xml_host_for_windows_wef = WinEventRememberHost, WinEventXmlHostOverride [(?::){0}WinEventLog:*] TRANSFORMS-1-SaveOrigChannel = WinEventSetOrigChannelName [(?::){0}XmlWinEventLog:*] TRANSFORMS-1-XmlSaveOrigChannel = WinEventSetOrigChannelName

transforms.conf

[WinEventHostOverride] DEST_KEY = MetaData:Host REGEX = (?m)ComputerName=(.*)?\b FORMAT = host::$1 [WinEventXmlHostOverride] DEST_KEY = MetaData:Host REGEX = <Computer>(.*).*?<\/Computer> FORMAT = host::$1 [WinEventRememberHost] SOURCE_KEY = MetaData:Host REGEX = host::(.+) FORMAT = host_UF::$1 WRITE_META = true [WinEventSetOrigChannelName] REGEX = WinEventLog:(.*) SOURCE_KEY = MetaData:Source FORMAT = original_channel::$1 WRITE_META = true