Hello,
I am trying to set up a heavy forwarder that filters the received Events from a universal forwarder and other sources, send them all to another logging solution per syslog, and only a subset of logs should be sent for further use to a Splunk indexer.
For now I am only experimenting with Windows event logs to get a hang of proper filtering, but it simply doesn't work.
Configs on the heavy forwarder in %Splunk_Home%\etc\system\local
Inputs.conf
#no code here
Because I defined the input port for the Splunk forwarder in the Splunk Web, but it would be nice to now in what .conf file Splunk saves this port.
Outputs.conf
[tcpout]
defaultGroup = nothing
[tcpout:OutSplunk]
server = <server>:<port>
[syslog:OutLogger]
server=<server>:<port>
type=tcp
props.conf
[WinEventLog:Security]
TRANSFORMS-route=WinSecEvent-Splunk,Everything
[WinEventLog:Setup]
TRANSFORMS-route=AllWinEvent-Splunk,Everything
[WinEventLog:System]
TRANSFORMS-route=AllWinEvent-Splunk,Everything
[WinEventLog:Application]
TRANSFORMS-route=AllWinEvent-Splunk,Everything
transforms.conf
[WinSecEvent-Splunk]
REGEX=(?msi)(.*Keywords=(?!Audit\sSuccess).*)
DEST_KEY=_TCP_ROUTING
FORMAT=OutSplunk
[AllWinEvent-Splunk]
REGEX=(?msi)(.*Type=(Error|Warning).*)
DEST_KEY=_TCP_ROUTING
FORMAT=OutSplunk
[Everything]
REGEX=.
DEST_KEY=_SYSLOG_ROUTING
FORMAT=OutLogger
The syslog output somehow works, but the filtering for the Windows events not. The indexer gets all events.
To check my regular expressions I use https://regex101.com . Since splunk also uses perl compatible regular expressions (pcre), it should work in theory, but in practice the events don't get filtered.
Update 03.08.2015
11:00 AM
Instead of creating a new question I will ask it here
according to this site
http://docs.splunk.com/Documentation/Splunk/6.2.4/Forwarding/Forwarddatatothird-partysystemsd#Syslog_data
the syslog processor of splunk creates rfc 3164 compliant output even with windows event logs but my third-party logging solution receives every line of a win event as a syslog message and not the the whole event as one message.
I will try to contact the support / a splunk partner about it but maybe someone here knows the answer
And by the way the filtering still doesn't work even after playing around a bit with the regex
@maciep
you were right, I found the config for the splunk input set in the web gui in an app
it is in %Splunk_Home%\etc\apps\search\local\input.conf
01:00 PM
@MuS
thanks for the answer that solved the problem with the filtering
I thought that you define a default group for events and messages where you don't have routing rules, so that everything that doesn't have a stanza in props.conf and transforms.conf still gets routed to an default indexer.
By the way all data has to be routed to the third party system and a copy of a subset to splunk
Ok now I still have the problem with the syslog output that is s clearly not rfc 3164 conform
I tested it now with two different third party systems and in both every line of an windows event is received as one syslog message.
Update 04.08.2015
9:00 AM
@maciep
that works by adding this line to every stanza of the windows events
SEDCMD-rmlines=s/[\n\r\t]/ /g
I get the events as one liner to the third party system but now I also get every thing as one line to splunk
is there maybe a way to remove the line breaks in the Everything stanza of transforms.conf or do I really need a another splunk instance to convert windows event logs to syslog before forwarding the message because the syslog processor doesn't create proper output
thanks in advance for the help
Ludwig_mdc
... View more