So in some unknown, previous version of Splunk we were able to forward syslog to an Rsyslog machine. The syslog messages began with the date/time and source host/IP of the message. This data was in the standard sysklogd format. It allowed us to write it out straight to a file and have it look like Rsyslog was the one receiving the messages. Current versions do not seem to include this header.
Is there any way to restore this behavior? Our goal is to, at a minimum, get any syslog data that is received by Splunk to be forwarded to our instance of Rsyslog with the original host that sent the data included in the log line.
I'm currently running version 4.0.8. I believe that the previous version was "current" around 2009-08-20.
If you were sending syslog data to rsyslog without splunk declaring itself in a header like a normal syslogd forward mechanism, you were basically "getting lucky" by having splunk pass along whatever syslog data it receieved over tcp and having that look "syslog enough" to rsyslog.
That is, in the past there was no specific support for emitting syslog data over udp or tcp. If you can still arrange to have only syslog-formatted events forwarded, and over TCP, I would expect it to work to the same degree that it it did before. If non-syslog-formatted data is involved then you have to take care not to forward that data, since it won't look anything like syslog to rsyslog.
Syslog servers when they operate in a tiered deployment, e.g. syslog to syslog server. This type of header nesting also occurs as per the RFC 3164. Splunk is acting the same way as a Syslog server would.
http://wiki.splunk.com/Community:Test:How_Splunk_behaves_when_receiving_or_forwarding_udp_data
Hello Dan,
Did you resolve this issue? I am experiencing it as well.
Thanks, Chris
If you were sending syslog data to rsyslog without splunk declaring itself in a header like a normal syslogd forward mechanism, you were basically "getting lucky" by having splunk pass along whatever syslog data it receieved over tcp and having that look "syslog enough" to rsyslog.
That is, in the past there was no specific support for emitting syslog data over udp or tcp. If you can still arrange to have only syslog-formatted events forwarded, and over TCP, I would expect it to work to the same degree that it it did before. If non-syslog-formatted data is involved then you have to take care not to forward that data, since it won't look anything like syslog to rsyslog.
You can define following in outputs.conf to forward data in syslog format.
[syslog]
defaultGroup=myGroup
[syslog:myGroup]
server = syslogserver.mycompany.com:514
This will send events out in syslog format via UDP to port 514 on syslogserver.mycompany.com. You can set 'type=tcp' in [syslog:mygGroup] stanza to send data out via TCP. You can add priority header if missing in the source by setting 'priority' field. The value set in priority field will be emitted in the syslog header.
I'm going to file an issue with Support about this...
Forwarding works fine. The real question is "Is there a way to maintain the source IP of the UDP syslog packet when forwarding to a 3rd party syslog listener?"
I already have it setup to forward to a remote syslog server. The problem is getting it to include the origin IP in the message.