topic Timestamp problem in Getting Data In

Timestamp problem

dhs_harry08 — Mon, 28 Sep 2020 13:59:21 GMT

Hi,

I have scheduled a cron in unix for every one minute which takes the uptime command output and writes to a file. It has been over 4 weeks now and I am observing now that every day splunk timestamp changes abruptly at 16:29.

You can see from below logs the event after 2013-05-27T16:29:00.000+0530 is coming as 2013-05-28T04:30:00.000+0530 and because of this a lot of events are missing and not getting reported.

1:00pm up 146 days 1:10, 1 user, load average: 2.11, 2.27, 3.20 2013-05-28T04:30:00.000+0530 1 28 0 may tuesday 2013 local nix-all-logs byk300lin241 3.2 2.11 2.27 /home/tkaushal/load.log unix_load 6 2

12:59am up 145 days 13:09, 0 users, load average: 0.24, 0.84, 1.15 2013-05-27T16:29:00.000+0530 12 27 59 may monday 2013 local nix-all-logs byk300lin241 1.15 0.24 0.84 /home/tkaushal/load.log unix_load 6 1

Please help me understand why is this ocuring and solution for it.

Regards,
Harish

Re: Timestamp problem

okrabbe_splunk — Tue, 28 May 2013 13:57:32 GMT

Are you relying on Splunk's automatic timestamp recognition?

If so, it is often more efficient and you can be certain it works correctly if you manually specify the timestamp recognition in props.conf. It should only take a few minutes.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

Re: Timestamp problem

dhs_harry08 — Tue, 28 May 2013 14:12:50 GMT

Yes I am relying on automatic timestamp recognition. But this is happening for only one sourcetype. All the others are fine.
If I change the timestamp will it not affect the others. Also is that the only option I got to correct it.

Re: Timestamp problem

bmacias84 — Mon, 28 Sep 2020 13:59:24 GMT

I see your event has two timestamps within the data, 1:00pm and 2013-05-28T04:30:00.000+0530. Splunk can sometimes get confused if you are using automatic timestamp recognition. I recommend configuring timestamp reconnition MAX_TIMESTAMP_LOOKAHEAD, TIME_PREFIX, TIME_FORMAT for this source. Configuring these can increase index peformance.

Re: Timestamp problem

okrabbe_splunk — Wed, 29 May 2013 02:27:41 GMT

Harish,

You should try something like this in props.conf

[mysourcetype]
MAX_TIMESTAMP_LOOKAHEAD=100
TIME_PREFIX = ^(\S+\s+){12}
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z

You could also use a source stanza (your log file name) instead but sourcetype is really a better way to do it.

I did not test this but it should be close. The link I posted earlier goes into a lot of detail around this topic:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

Re: Timestamp problem

dhs_harry08 — Wed, 29 May 2013 08:00:41 GMT

I will try this out. Thanks okrabbe