We have a requirement to send audit logs from our host servers (/var/log/audit/audit.log) to both our indexers and to a 3rd party host syslog server. I am testing with a host gary-test2.ussl.uhs with audit logs in /var/log/audit/audit.log.
I have configured the universal forwarder host gary-test2.ussl.uhs to redirect all it's logs to the heavy forwarder. I would like to have the heavy forwarder send it's logs to the indexers but also a copy of all audit events to the syslog server syslogp01.ussl.uhs
Here is the architecture involved with the routing.
Universal Forwarder
gary-test1.ussl.uhs
Heavy Forwarder
ussl-splkhfwt01.ussl.uhs
Indexers
splkidxt01.ussl.uhs
splkidxt02.ussl.uhs
Syslog server
syslogp01.ussl.uhs (10.17.8.206)
Here is how I configured the Heavy forwarder...
splkhfwt01.ussl.uhs (heavy forwarder host)
/opt/splunk/etc/apps/forwarder_syslog/local/props.conf
[source::/var/log/audit/audit.log]
TRANSFORMS-routing=troutingrsa
/opt/splunk/etc/apps/forwarder_syslog/local/transforms.conf
[troutingrsa]
REGEX=.
DEST_KEY=_SYSLOG_ROUTING
FORMAT=Myroutingrsa
/opt/splunk/etc/system/local/outputs.conf
[tcpout]
defaultGroup = default-autolb-group
indexAndForward = 0
[tcpout:default-autolb-group]
disabled = false
server = splkidxt01.ussl.uhs:9997,ussl-splkidxt02.ussl.uhs:9997
[syslog:Myroutingrsa]
server = 10.17.8.206:514
sendCookedData = false
type = udp
disabled = false
What I am seeing is that the /var/log/audit/audit.log logs from host gary-test2.ussl.uhs are appearing in search queries on Splunk. Also those same logs are appearing on the syslog server. Here is the problems I found. Logs other than the audit.log logs from the host gary-test2.ussl.uhs are also appearing on the syslog server. I suspected that the props.conf and the transforms.conf were not doing their job so I remarked out all the settings in props.conf and transforms.conf then restarted splunk. I found that the logs continued to be sent to the syslog server which says the props.conf and transforms.conf files are having no effect. Just to be sure, in the outputs.conf, I removed the "[syslog:Myroutingrsa]" and it's settings. That then made the logs stop forwarding to the syslog server. Does anyone see what is wrong with my forwarding configuration settings?
When you create the [syslog:Myroutingrsa] stanza, all data is sent to that group.
In order to avoid default sending data to that group, you could try something like
[syslog]
defaultGroup = bogusGroup
where bogusGroup does not exist.
this is analogous to the tcp routing instructions here: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Index_one_input_l...
Since the defaultGroup is set to the non-existent group "noforward" (meaning that there is no defaultGroup), the forwarder only forwards data that has been routed to explicit target groups in inputs.conf. It drops all other data.