Archive

TImestamps problem

Explorer

Hi Im running the newest splunk, with syslog-ng fifo pipe as a source and logs are coming from around the globe, splunk is in the US so when logs from China are hitting splunk they are like 10h ahead, and they dont show up in search till splunk riches that hour itself

Jun 8 23:37:39 tok-* SYST: Port 29 link active 100Mbs FULL duplex

Jun 8 20:07:40 10.115.1.2 SNTP: The SNTP server parameter value (pool.ntp.org) can not be resolved.

Jun 8 10:37:47 del-## Jun: 8 20:05:42 netTool.sntp: : Failed to sntp request to server 10.**

as you can see logs are coming with local times, and they get indexed like that, now time on the splunk machine is 10:37 and last log shows 2 time zones, i dont have source in props.conf b/c i dont use files to import the logs all i have is syslog pipe and splunk set up to

[fifo:///var/syslog-ng/syslog_fifo]

disabled = false

host = MYHoST

sourcetype = syslog

how can I change that so all of the logs would be logged with 2 timezones, or just logged with the splunk local time instead of sender local time ? thanks

Tags (1)

Path Finder

Did this work? it sounds like you have a similar issue to ours. One way to stop the timestamp from being auto adjusted by the search head is to change the TZ at index time in the props file to that of the search head location. This does have side effects though as the local data will be theoretically indexed incorrectly.

0 Karma

Path Finder

It seems for now that there is no way of disconnecting the TZ from a timestamp. I have effectively designed the architecture to resolve this issue. There will be a different instant of the application per region and timestamps will be indexed with the correct TZ to allow future scalability...

0 Karma

Splunk Employee
Splunk Employee

If you just want to use arrival time for this source rather than extracted time you can set in props.conf:

[source::/var/syslog-ng/syslog_fifo]
DATETIME_CONFIG = CURRENT

If you can detangle the different timezones into different files, you can set in props.conf:

[source::/var/syslog-ng/host1/syslog_fifo]
TZ = <host1 timezone>

[source::/var/syslog-ng/host2/syslog_fifo]
TZ = <host2 timezone>