Why are log sent to Splunk in GMT displayed in the future?

[zliu]

Several devices that only support sending logs out stamped with GMT and splunk displays them in the future.

Placed below in $SPLUNK_HOME/etc/system/local/props.conf

[host::snort*]
TZ = GMT

[woodcock]

What is the "date_zone" value for your indexed events? If your setting is in effect, it should be "0000" or "UTC" or "GMT" but DEFINITELY NOT "local". My suspicion is that you will find it is "local", which tells you that your TZ setting is not in effect. Here are a few reasons this might be so:
1: If you are doing any kind of host-override (including "host_segment"), props.conf will trigger off the ORIGINAL host value, not the new host value; you might be able to use [source::] instead of [host::]
2: You might not be deploying the props.conf file to your indexers which is where this configuration file needs to be in order to function (NOTE: there is a twist to this as of v6.0 in that a props.conf TZ setting on the forwarder will be honored and have precedence over the same setting on the indexer)

[gkanapathy]

If the Splunk indexer's clock is correct, and the Splunk indexer's time zone is correct, then in fact all you should need to do it tell Splunk what the time zone of your incoming data is. Splunk will do the appropriate corrections.

If your stanza:

[host::snort*]
TZ=GMT

is not working, it is likely because either you have configured it in the wrong place (it must be where the parsing is done: http://www.splunk.com/wiki/Where_do_I_configure_my_Splunk_settings%3F), or because the host name does not (yet) match. e.g., for syslog inputs, the host value may not match what you see in Splunk because Splunk has transformed it.

[gkanapathy]

Are you saying that the [host::snort*] stanza you used is not working to adjust the time appropriately? Please clarify.

[Mick]

Splunk extracts timestamps from events and indexes them in epoch format, and uses UTC time (Coordinated Universal Time). To see what time Splunk has applied to a particular event, check out the '_time' field.

When you apply an offset, Splunk will determine the timezone that your server is running in and increase/decrease the '_time' value appropriately. When searching this data, Splunk will display the time relative to your current location. Eg, an event from 16:00 GMT will be displayed as 08:00 PST

If your data is appearing with a time in the future next to it, it's possible that the indexing server is set to the wrong timezone, or perhaps Splunk is extracting a different timestamp than you expect it to.

If you can supply an example of the event time vs. the indexed time, and maybe a sample event, perhaps we can narrow this down further. Always provide as much information as possible with your question, the less we go back & forth the better.

[Chris_R_]

lulz at typing simultaneous response

[Chris_R_]

Hey Splunker_J, Which timezone is the forwarding device and indexer in?

it's possible the snort timestamps are just not being parsed correctly. Do you have any sample event data from them? You might have to specify the time string with:

TIME_PREFIX = < some characters >
TIME_FORMAT = %d/%b/%Y:%H:%M:%S this would match a date like 03/Feb/2010:12:35:11

(For TIME_FORMAT syntax use a reference for strptime such as http://www.opengroup.org/onlinepubs/009695399/functions/strptime.html