topic Re: forward data to a syslog server in Getting Data In

forward data to a syslog server

acalvo — Fri, 29 Oct 2010 22:49:19 GMT

We're trying to forward data to a syslog server from a splunk server. However, seems that the hostname and process id tags are missing.

We've set up a central splunk server (10.10.40.9), and another splunk server to forward some data to a syslog server (10.10.40.10).

Configurations used: outputs.conf:

[tcpout]
defaultGroup = 10.10.40.9_9997

[tcpout:10.10.40.9_9997]
server = 10.10.40.9:9997

[tcpout-server://10.10.40.9:9997]

[syslog:localhost_10514]
server = 127.0.0.1:10514
type = tcp

inputs.conf:

[monitor:///var/log]
disabled = false

props.conf:

[host::*]
TRANSFORMS-routing=SYSLOG_FWD

transforms.conf:

[SYSLOG_FWD]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = localhost_10514

example output messages (forwarded splunk output to syslog):

2010-10-29T17:13:05.557347+02:00 10-29-2010 17: 13:04.924 INFO  Metrics - group=per_source_thruput, series="/opt/splunk/var/log/splunk/splunkd.log", kbps=2.094271, eps=20.466667, kb=62.828125
2010-10-29T17:13:05.557347+02:00 10-29-2010 17: 13:04.924 INFO  Metrics - group=per_source_thruput, series="/opt/splunk/var/log/splunk/splunkd_stderr.log", kbps=0.000684, eps=0.033333, kb=0.020508

example desired output:

2010-10-23T08:29:08.087021+02:00 VCS02 glassfish-log: [#|2010-10-23T08:29:00.584+0200|INFO|glassfish3.0.1|null|_ThreadID=29;_ThreadName=Thread-1;|Total number of available updates : 0|#]
2010-10-24T01:38:48.570726+02:00 VCS02 ntpd[14162]: synchronized to 77.226.252.14, stratum 2
2010-10-24T04:02:08.609446+02:00 VCS02 rsyslogd: [origin software="rsyslogd" swVersion="5.6.0" x-pid="26020" x-info="http://www.rsyslog.com"] rsyslogd was HUPed

Re: forward data to a syslog server

ftk — Fri, 29 Oct 2010 23:06:39 GMT

What do you want the output to look like?

Re: forward data to a syslog server

thepocketwade — Sat, 30 Oct 2010 00:01:25 GMT

Have you considered putting the syslog server in front of the Splunk server?

Re: forward data to a syslog server

acalvo — Sat, 30 Oct 2010 05:45:22 GMT

The message should look like a standard syslog message (while I was testing I've used the same log that Splunk produces to send it to the syslog server):

$DATE $HOSTNAME $PROCESS[$PID] $MESSAGE

Re: forward data to a syslog server

calyope7 — Wed, 13 Apr 2011 04:47:50 GMT

Hello acalvo,

Did you find a resolution to this issue?

Thanks, Chris

Re: forward data to a syslog server

nisse — Wed, 03 Aug 2011 20:02:50 GMT

Re: forward data to a syslog server

hexx — Mon, 28 Sep 2020 10:21:50 GMT

In order to receive RFC3164-compliant syslog output from Splunk (and namely, to include a time stamp in the sent event), you need to make sure to adequately set the timestampformat configuration key in the [syslog:] stanza in outputs.conf, as per outputs.conf.spec:

timestampformat = <format>
* If specified, the formatted timestamps are added to the start of events forwarded to syslog.
* As above, this logic is only applied when the data is not syslog, or the syslogSourceType.
* The format is a strftime-style timestamp formatting string. This is the same implementation used in the 'eval' search command, splunk logging, and other places in splunkd.
    *  For example: %b %e %H:%M:%S
    * %b - Abbreviated month name (Jan, Feb, ...)
    * %e - Day of month
    * %H - Hour
    * %M - Minute
    * %s - Second
* For a more exhaustive list of the formatting specifiers, refer to the online documentation.
* Note that the string is not quoted.
* Defaults to unset, which means that no timestamp will be inserted into the front of events.

Test configuration (tested with Splunk 4.3 only😞

outputs.conf:

[syslog:syslog_out] server = syslog.splunk.com:514 type = tcp timestampformat = %b %e %H:%M:%S

props.conf:

[syslog_test] TRANSFORMS-routing = syslog_routing

transforms.conf:

[syslog_routing] REGEX = . DEST_KEY = _SYSLOG_ROUTING FORMAT = syslog_out

With this configuration, all events with sourcetype "syslog_test" will be routed to syslog host "syslog.splunk.com" on port TCP/514. The default priority code of "13" will be used, which is equivalent to "user.info", and a time stamp will be added as a prefix.

Example:

On the server hosting the Splunk instance:

[root@beefysup01 43]# cat ../sources/test.log This is a sample syslog event [root@beefysup01 43]# $SPLUNK_HOME/bin/splunk add oneshot ../sources/test.log -sourcetype syslog_test Oneshot '/home/octavio/sources/test.log' added

On the server recipient of the syslog forwarded messages, using netcat to receive the events:

[root@syslog.splunk.com:/]# nc -kl 514 <13>Jan 25 19:52:07 beefysup01 This is a sample syslog event

Note: I was unable to make this configuration work in versions earlier to Splunk 4.3. There are reports that this specific feature can fail in certain scenarios in 4.2.x. If you need this feature to work, please upgrade to Splunk 4.3.