Why is there a 4 hour time difference between event timestamp vs _time?

[iamsplunker]

Hi Splunkers, 

I have an issue with the timestamp the data is being indexed. Here is an example of my logs.

I applied the props at sourcetype level. However it doesn't seem to be working- Please Help

Scenario -1

Time                                                                  Event

6/20/22  10:35:59.833 PM               2022-06-20 18:35:59,833  [200] Error logs http client 

props.conf

TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N

TIME_PREFIX = ^

MAX_TIMESTAMP_LOOKAHEAD = 24

TZ = UTC

Scenario - 2

Time                                                                  Event

6/20/22 10:24:05.000 PM                  2022-06-20 22:23:53 Error logs http client 

[gcusello]

Hi @iamsplunker,

to help you in debugging your problem I need to have a sample of your logs, anyway, you have to exactly identify TIME_FORMAT and TIME_PREFIX.Ciao.

Giuseppe

[iamsplunker]

@gcusello : Here is the sample 

Time                                                               Event

6/23/22  9:52:26.000 PM                10.100.200.10 - - [23/Jun/2022:17:52:26 -0400] TAIL/mcquote/ApplStatusServers HTTP/2.0 900 - "-" "-" - -

6/23/22  9:52:26.000 PM                20.100.200.20 - - [23/Jun/2022:17:52:26 -0400] TAIL/mcquote/ApplStatusServers HTTP/2.0 900 - "-" "-" - -

[gcusello · Accepted solution]

Hi @iamsplunker,

I suppose that the timestamp you want is the date between square brackets.

In this case you have to use:

[your_sourcetype]
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
TIME.PREFIX = \[

please check if before the timezone there's a space.

Ciao.

Giuseppe

[gcusello]

Hi @iamsplunker,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

[iamsplunker]

@gcusello Looks Like it's not working . I also added TZ and MAX_TIMESTAMP_LOOKAHEAD

[sourcetype]
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
TIME_PREFIX = \[
MAX_TIMESTAMP_LOOKAHEAD = 27
TZ = UTC

[isoutamo]

When your timestamp contains TZ information like -0400, you shouldn't add any TZ information to props.
As @gcusello said, you must add props.conf into the 1st full splunk enterprise instance HF or Indexer from source system.

[gcusello]

Hi @iamsplunker,

at first please check if there's a space between seconds and timezone as it seems.

Then, where do you located the pros.conf?

it must be located on Indexers or (if present) on Heavy Forwarders.

Ciao.

Giuseppe

[iamsplunker]

@gcusello @isoutamo 

Yes, There is a space between seconds and timezone so the time format is correct I believe. I removed the TZ from my props.

When I tested the data with TIME_PREFIX and TIME_FORMAT it looks like the event time is highlighted(as below) however it doesn't seem to take that timestamp.

Time                                                      Event

6/28/22 2:44:30.000 PM          11.146.180.90 - - [28/Jun/2022:10:44:30 -0400] GETS /rebel/Frontend

[isoutamo]

Internally Splunk use UTC and it shows it on your local TZ or what ever you have configured on your GUI. See <Your Account Name> -> Preferences -> Time Zone.