topic Re: Syslog forwarding and rewrite of host in Getting Data In

Syslog forwarding and rewrite of host

approachct — Thu, 09 Jan 2014 19:22:50 GMT

I am forwarding events from a group of servers to an Indexer by way of a Splunk light forwarder. I have forwarding turned on the Indexer to send these events to a syslog server, however when the syslog server receives the forwarded Splunk data from the Indexer its shows the host as the indexer name not the original host

Light Forwarders (eventdata) -> Indexer -> Syslog

What is the quickest way to rewrite the source host name in the syslog so that it correctly show the original host?

Re: Syslog forwarding and rewrite of host

lguinn2 — Thu, 09 Jan 2014 19:48:20 GMT

If the sourcetype of the incoming data is set to syslog, Splunk should do this by default.

If it is not, you can do it yourself this way, placing props.conf and transforms.conf on the indexer:

props.conf

[yoursourcetypehere]
TRANSFORMS-sethost = set-host

transforms.conf

[set-host]
DEST_KEY = MetaData:Host
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(\w[\w\.\-]{2,})\]?\s
FORMAT = host::$1

BTW, I didn't write that regular expression, I copied it from the etc/system/default/transforms.conf, so it is the same one that Splunk uses for syslog.

Re: Syslog forwarding and rewrite of host

yungro — Tue, 21 Jan 2014 15:52:20 GMT

How should the outputs.conf be configured with props.conf and transforms.conf configured as above?

Re: Syslog forwarding and rewrite of host

dvb — Fri, 31 Oct 2014 09:23:23 GMT

Lisa, I think you misunderstood the question. The question wasn't about setting the host field correctly in splunk, but about changing the syslog header when forwarding the data. I have the same problem and haven't yet found a solution...
...let me make an example.

Setting:
Maschine Forwarder ---> Indexer --> syslog host
Log / Event (1) (2) (3)

During its travel from (1) to (3), a log line looks as follows:
(1) Oct 30 08:25:43 xxxx yyyy.zzzz aaaa
(2) Oct 30 08:25:43 xxxx yyyy.zzzz aaaa
_time=Oct 30 08:25:43 host=xxxx sourcetype=syslog ...
(3) Oct 30 08:25:45 mmm nnn.ooo Oct 30 08:25:43 xxxx yyyy.zzzz aaaa

xxxx: original host
mmm: Indexer hostname
yyyy.zzzz: original facility.priority
nnn.ooo: facility.priority set in outputs.conf of mmm for syslog forwarding
aaaa: original message

This means that all data being received by the syslog host seems to be originating from the Splunk Indexer and has the same facility.priority (if one is not using different stanzas in outputs.conf, e.g. for different source types).
In the case of (1) being syslog, the syslog host can parse the message and take the original hostname (xxxx) and facility.priority (yyyy.zzzz) out of the message.
But if (1) is not containing a hostname, the original host cannot be induced from (3). For this it would be needed that the Splunk Indexer, when forwarding data by syslog, adds a syslog header which contains the value of the host field (per event) instead of its own hostname.

Re: Syslog forwarding and rewrite of host

dvb — Mon, 28 Sep 2020 18:13:55 GMT

Using the option syslogSourceType you can tell Splunk which source types are already in syslog format. For these source types the syslog header will then contain the hostname of the original log (and not the hostname of the intermediate forwarder). Unfortunately the option doesn't accept regex, so multiple output stanzas are needed (see example) if your syslog source types have no common subset. Further it seems that this option has no influence on facility.priority, i.e. facility.priority will always be user.notice instead of the original one.
--> I filed an enhancement request to change syslogSourceType to accept regex.

===inputs.conf===
[splunktcp-ssl://9997]
connection_host = ip
_SYSLOG_ROUTING = tacLOG_1515a

[monitor:///data/Logweiche/mmm]
index=nnn
sourcetype=ooo
_SYSLOG_ROUTING = tacLOG_1515a

[monitor:///data/Logweiche/xxx]
index=yyy
sourcetype=zzz
_SYSLOG_ROUTING = tacLOG_1515b

===outputs.conf===
[syslog]
[syslog:tacLOG_1515a]
type = tcp

set no additional syslog header for sourcetype ooo*

syslogSourceType = ooo

[syslog:tacLOG_1515b]
type = tcp

no additional header for sourcetype zzz

syslogSourceType = sourcetype::zzz

Re: Syslog forwarding and rewrite of host

goelli — Mon, 19 Aug 2019 12:53:45 GMT

Hi, yo you have any information, if this Enhancement Request was implemented?

Re: Syslog forwarding and rewrite of host

dvb — Wed, 21 Aug 2019 06:51:00 GMT

No, I don't have any information.

Re: Syslog forwarding and rewrite of host

goelli — Wed, 21 Aug 2019 09:12:10 GMT

I opened a case and it's confirmed, that the ER was not implemented by now. Our need is now added to ER SPL-175134, but no big hope for having this implemented soon.