topic Re: props.conf and syslog in Getting Data In

props.conf and syslog

trumpjk — Mon, 07 Jan 2013 18:40:36 GMT

I have an entry in props.conf for TZ conversion i.e:

[host::SOMEHOST]
TZ = UTC

If the host is running the universal forwarder the splunk indexer with handle the conversion and display entry using local time.

If I forward logs from host via syslog the indexer does not do the time conversion.

Any suggestions?

Re: props.conf and syslog

emiller42 — Mon, 07 Jan 2013 19:45:20 GMT

So I'm interpreting this as two separate configurations:

Configuration 1:

Forwarder on external host, forwarding to Splunk indexer.
Splunk indexer has props.conf stanza for that host name with a TZ setting.

Configuration 2

Syslog on external host sending lines to indexer.
Indexer is monitoring syslog locally to index lines.
Same props.conf stanza as above.

In the second configuration, the props.conf stanza won't apply as it's specifying the remote host, while the syslog data is being monitored locally (and thus would have the indexer as the host)

In the second scenario, I would recommend using sourcetype or source stanzas to apply the TZ setting, not host. As setting it for the indexer would apply the TZ setting to everything indexed locally. (Including _internal logs) Instead, create a source stanza or sourcetype stanza. (Whichever is most appropriate) and use that.

[source::syslog]
TZ = UTC

if your source name is different, then change accordingly.

Re: props.conf and syslog

trumpjk — Mon, 07 Jan 2013 19:51:59 GMT

To clarify: I edited the props.conf on the indexer. The remote host(s) is forwarding syslog to indexer via syslog-ng. Indexer is receiving logs. I have over 200 systems forwarding via syslog and not all are using same TZ. SOme are EST and some are GMT

Re: props.conf and syslog

emiller42 — Mon, 07 Jan 2013 20:12:17 GMT

That makes things more complicated. You're going to want to use a transform to get the appropriate host values out of the individual log lines, then host:: stanzas with the relevant TZ settings. However, I'm not sure if this can be done in one props.conf, or if a heavy forwarder is needed. (HF does the transform portion before forwarding to the indexer)

Can't syslog-ng include timezone in the timestamp of the logs? That would take care of things too. (Although that might not be something you control)

Re: props.conf and syslog

emiller42 — Mon, 07 Jan 2013 20:13:09 GMT

see the following link for an example of using transforms on syslog data:

http://splunk-base.splunk.com/answers/60972/split-syslog-input-into-multiple-indexes

Re: props.conf and syslog

trumpjk — Tue, 08 Jan 2013 16:16:12 GMT

I was able to add a template to the syslog-ng.conf file to modify date/time stamp to include timezone offset. This solved my problem. Thanks for your help.