First of all, can UF's send syslog to a third party? The documentation says, "You can configure a forwarder" but does not specify. It also says "You can also filter the data with props.conf and transforms.conf (heavy forwarder only)". That is NOT what i want to do. The UF is already forwarding to an Indexer, now i need it to send everything to a third party IDS via syslog. I've tried variations of the example
here but haven't had any luck. I think my main issue is not fully understanding the relationship between props, transforms, and outputs.
[syslog:what_should_go_here?] server=10.x.x.x:514 type=udp
I need to send everything to the IDS and don't want to do any filtering, therefore i don't think i need a transform. But do i still need to make a group in props?
As an FYI: As of Splunk version 6.0, documentation clearly states that only a heavy forwarder can forward data in the `syslog' format.
What about putting an outputs.conf on the indexers to forward data out VIA syslog. The universal forwarders don't do any filtering or anything, and might not even be capable of doing syslog output.
Thanks for the response. Setting an outputs.conf on the Indexer is an option, but wanted to explore this first as it could reduce cpu/bandwidth on the Indexer. Once again, the Doc doesn't explicitly say that UF does not send syslog. It only says that you need a heavy forwarder to do any filtering which I am not trying to accomplish.
There is a very severe issue with forwarding to a 3rd party syslog from any full Splunk instance (SH, HF, Indexer, etc.).
Let' say you want to send data to a 3rd party syslog via TCP and it happens that syslog is unreachable (not receiving data for whatever reason (service down, network down, etc.). After some time all queues start to fill up from output (e.g. output queue for syslog target group is just 97B and you cannot change it) over parsing (parsing, merging, typing, ...) to input queues. Once input queue is full Splunk instance will stop receving data - on indexer you effectively STOP indexing. This is by design.
There are some solutions for this - none is great.
1. Send to a 3rd party syslog via UDP - not reliable and for many customer unacceptable.
2. Send requrested content via some other "tool". If you want to send out locally monitored files you can use SyslogNG or Cribl if the target destination is syslog-capable consumer.
BTW Cribl is the only non-Splunk solution out on the market that supports native input/output for Splunk proprietary S2S protocol. So you can collect data by UF and send to Cribl for processing and forward further to Splunk, Syslog and many other targets.
You cannot solve this puzzle with Splunk only ideally.