Getting Data In

How to Forward events to both our indexers and a syslog server?

giga444
Engager

We have a requirement to send audit logs from our host servers (/var/log/audit/audit.log) to both our indexers and to a 3rd party host syslog server. I am testing with a host gary-test2.ussl.uhs with audit logs in /var/log/audit/audit.log.

I have configured the universal forwarder host gary-test2.ussl.uhs to redirect all it's logs to the heavy forwarder. I would like to have the heavy forwarder send it's logs to the indexers but also a copy of all audit events to the syslog server syslogp01.ussl.uhs

Here is the architecture involved with the routing.
Universal Forwarder
gary-test1.ussl.uhs

Heavy Forwarder
ussl-splkhfwt01.ussl.uhs

Indexers
splkidxt01.ussl.uhs
splkidxt02.ussl.uhs

Syslog server
syslogp01.ussl.uhs (10.17.8.206)

Here is how I configured the Heavy forwarder...
splkhfwt01.ussl.uhs (heavy forwarder host)

/opt/splunk/etc/apps/forwarder_syslog/local/props.conf
[source::/var/log/audit/audit.log]
TRANSFORMS-routing=troutingrsa

/opt/splunk/etc/apps/forwarder_syslog/local/transforms.conf
[troutingrsa]
REGEX=.
DEST_KEY=_SYSLOG_ROUTING
FORMAT=Myroutingrsa

/opt/splunk/etc/system/local/outputs.conf
[tcpout]
defaultGroup = default-autolb-group
indexAndForward = 0

[tcpout:default-autolb-group]
disabled = false
server = splkidxt01.ussl.uhs:9997,ussl-splkidxt02.ussl.uhs:9997

[syslog:Myroutingrsa]
server = 10.17.8.206:514
sendCookedData = false
type = udp
disabled = false

What I am seeing is that the /var/log/audit/audit.log logs from host gary-test2.ussl.uhs are appearing in search queries on Splunk. Also those same logs are appearing on the syslog server. Here is the problems I found. Logs other than the audit.log logs from the host gary-test2.ussl.uhs are also appearing on the syslog server. I suspected that the props.conf and the transforms.conf were not doing their job so I remarked out all the settings in props.conf and transforms.conf then restarted splunk. I found that the logs continued to be sent to the syslog server which says the props.conf and transforms.conf files are having no effect. Just to be sure, in the outputs.conf, I removed the "[syslog:Myroutingrsa]" and it's settings. That then made the logs stop forwarding to the syslog server. Does anyone see what is wrong with my forwarding configuration settings?

0 Karma

ruman_splunk
Splunk Employee
Splunk Employee

When you create the [syslog:Myroutingrsa] stanza, all data is sent to that group.

In order to avoid default sending data to that group, you could try something like

[syslog]
defaultGroup = bogusGroup

where bogusGroup does not exist.

this is analogous to the tcp routing instructions here: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Index_one_input_l...

Since the defaultGroup is set to the non-existent group "noforward" (meaning that there is no defaultGroup), the forwarder only forwards data that has been routed to explicit target groups in inputs.conf. It drops all other data. 
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

🍂 Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...