topic Re: Update: problem with syslog output [Solved]filter and route Windows event logs in Getting Data In

Update: problem with syslog output [Solved]filter and route Windows event logs

Ludwig_MDC — Thu, 30 Jul 2015 12:28:29 GMT

Hello,

I am trying to set up a heavy forwarder that filters the received Events from a universal forwarder and other sources, send them all to another logging solution per syslog, and only a subset of logs should be sent for further use to a Splunk indexer.

For now I am only experimenting with Windows event logs to get a hang of proper filtering, but it simply doesn't work.
Configs on the heavy forwarder in %Splunk_Home%\etc\system\local

Inputs.conf

#no code here

Because I defined the input port for the Splunk forwarder in the Splunk Web, but it would be nice to now in what .conf file Splunk saves this port.

Outputs.conf

[tcpout]
defaultGroup = nothing

[tcpout:OutSplunk]
server = <server>:<port>

[syslog:OutLogger]
server=<server>:<port>
type=tcp

props.conf

[WinEventLog:Security]
TRANSFORMS-route=WinSecEvent-Splunk,Everything

[WinEventLog:Setup]
TRANSFORMS-route=AllWinEvent-Splunk,Everything

[WinEventLog:System]
TRANSFORMS-route=AllWinEvent-Splunk,Everything

[WinEventLog:Application]
TRANSFORMS-route=AllWinEvent-Splunk,Everything

transforms.conf

[WinSecEvent-Splunk]
REGEX=(?msi)(.*Keywords=(?!Audit\sSuccess).*)
DEST_KEY=_TCP_ROUTING
FORMAT=OutSplunk

[AllWinEvent-Splunk]
REGEX=(?msi)(.*Type=(Error|Warning).*)
DEST_KEY=_TCP_ROUTING
FORMAT=OutSplunk

[Everything]
REGEX=.
DEST_KEY=_SYSLOG_ROUTING
FORMAT=OutLogger

The syslog output somehow works, but the filtering for the Windows events not. The indexer gets all events.
To check my regular expressions I use https://regex101.com . Since splunk also uses perl compatible regular expressions (pcre), it should work in theory, but in practice the events don't get filtered.

Update 03.08.2015
11:00 AM
Instead of creating a new question I will ask it here
according to this site
http://docs.splunk.com/Documentation/Splunk/6.2.4/Forwarding/Forwarddatatothird-partysystemsd#Syslog_data
the syslog processor of splunk creates rfc 3164 compliant output even with windows event logs but my third-party logging solution receives every line of a win event as a syslog message and not the the whole event as one message.
I will try to contact the support / a splunk partner about it but maybe someone here knows the answer

And by the way the filtering still doesn't work even after playing around a bit with the regex

@maciep
you were right, I found the config for the splunk input set in the web gui in an app
it is in %Splunk_Home%\etc\apps\search\local\input.conf

01:00 PM

@MuS
thanks for the answer that solved the problem with the filtering
I thought that you define a default group for events and messages where you don't have routing rules, so that everything that doesn't have a stanza in props.conf and transforms.conf still gets routed to an default indexer.
By the way all data has to be routed to the third party system and a copy of a subset to splunk

Ok now I still have the problem with the syslog output that is s clearly not rfc 3164 conform
I tested it now with two different third party systems and in both every line of an windows event is received as one syslog message.

Update 04.08.2015

9:00 AM

@maciep

that works by adding this line to every stanza of the windows events

SEDCMD-rmlines=s/[\n\r\t]/ /g

I get the events as one liner to the third party system but now I also get every thing as one line to splunk
is there maybe a way to remove the line breaks in the Everything stanza of transforms.conf or do I really need a another splunk instance to convert windows event logs to syslog before forwarding the message because the syslog processor doesn't create proper output

thanks in advance for the help
Ludwig_mdc

Re: Update: problem with syslog output [Solved]filter and route Windows event logs

maciep — Thu, 30 Jul 2015 15:43:40 GMT

If you're asking about the receiving port for the heavy forwarder, check inputs.conf. But since you set it in the GUI, it will likely be in one of your apps - like launcher or search etc. It won't be in system/local.

For your config, I would think everything would only be getting sent to the syslog server, so I'm obviously no help there. I'm working on a similar project and have been struggling with routing from a heavy forwarder, so a lot to learn yet there for me. But I do have a ticket open with Splunk so if I gain a better understanding and can apply here as well, I'll follow at that time.

Re: Update: problem with syslog output [Solved]filter and route Windows event logs

MuS — Mon, 03 Aug 2015 09:48:30 GMT

Hi Ludwig_MDC,

Your Splunk sends everything to the third party server because you use the defaultGroup in your outputs.conf see the docs for details

defaultGroup = <target_group>, <target_group>, ...
* The forwarder sends all data to the specified groups.

cheers, MuS

Re: Update: problem with syslog output [Solved]filter and route Windows event logs

maciep — Mon, 03 Aug 2015 12:59:01 GMT

We got this from a vendor who I think got it from the comment section here: http://blogs.splunk.com/2013/10/14/windows-event-logs-in-splunk-6/

In our props on the heavy forwarder for this sourcetype, we have the following. It seems to format the data in a way that the target syslog server expects.

SHOULD_LINEMERGE = true
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{2,4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
REPORT-MESSAGE = wel-message, wel-eq-kv, wel-col-kv
KV_MODE=none
TRANSFORMS-FIELDS = strip-winevt-linebreaker
SEDCMD-win=s/(?mis)(Token Elevation Type indicates|This event is generated|Application Id=).*$//g
SEDCMD-rmlines=s/[\n\r\t]/ /g

Re: Update: problem with syslog output [Solved]filter and route Windows event logs

woodcock — Tue, 04 Aug 2015 15:01:18 GMT

It is unnecessary (maybe even problematic) to update your question's subject-line to indicate a solution. Just add the solution as an "Answer" (not a "Comment") and then click "Accept" on your Answer to close out the question and let others know that there is a validated solution.

Re: Update: problem with syslog output [Solved]filter and route Windows event logs

Ludwig_MDC — Wed, 05 Aug 2015 06:24:17 GMT

it works but you will also get single line windows events in the splunk indexer
therefore I am a bit disappointed that splunk an enterprise logging solution that claims in its documentation to be able to create rfc 3164 compliant syslog output even with event logs
´http://docs.splunk.com/Documentation/Splunk/6.2.4/Forwarding/Forwarddatatothird-partysystemsd#Syslog_data´
or does that only work if you have the heavy forwarder on every system instead of an universal forwarder

Re: Update: problem with syslog output [Solved]filter and route Windows event logs

campbellj1977 — Fri, 21 Dec 2018 20:26:31 GMT

I do not see an answer here. Why was it closed?