topic Re: Timestamp extraction is not working in Splunk Enterprise

Timestamp extraction is not working

iamvinaykumar — Wed, 16 Jun 2021 19:13:45 GMT

WE have data coming from syslog which is like below :

2021-06-16T19:03:02+02:00 XXXXXXXXXX - (6/16/21 5:03:02.000 PM - splunk extracted time)

2021-06-16T19:02:58+02:00 XXXXXXXXXX - (6/16/21 5:02:58.000 PM - splunk extracted time)

these data's are sent to indexer from syslog via heavy forwarders , data from syslog is actually in the UTC time zone but +2:00 have been appended wrongly in the syslog data , splunk intelligence is taking +2:00 and substracting it from utc time and there is 2 hour time gap ,

inorder to extract correct time stamp we placed props.conf in the heavy forwarder (config below) but still the extraction is not working . ensured that on disk config is coming as below but still required timestamps are not extracted , still there is time gap of 2 hours .

TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %Y-%m-%dT%H%M%S
LINE_BREAKER = ([\n\r]+)
SHOULD_LINEMERGE = false
TRUNCATE = 999999

Any solutions to extract highlighted time ?

Re: Timestamp extraction is not working

isoutamo — Wed, 16 Jun 2021 21:09:16 GMT

Have you try TZ=UTC on props.conf? I suppose that HFs own time zone is what it is using now even you cut of that +02:00 from event.
Another option is try use INGEST_EVAL = _time := .... setting first time with correct TZ and then do strptime for it when assign it to _time.
r. Ismo

Re: Timestamp extraction is not working

iamvinaykumar — Thu, 17 Jun 2021 14:24:12 GMT

@isoutamo Yes we tried to set TZ = UTC but still Splunk is reading from the log , Also tried to set the time zone in source i e universal forwarder forwarder in the syslog servers . but even that didn't helped .

Will index-time eval function work in this case ?

Re: Timestamp extraction is not working

isoutamo — Tue, 29 Jun 2021 20:02:30 GMT

Here is one example to use INGEST_EVAL for manipulating time https://www.tekstream.com/a-use-case-for-ingest-time-eval/

r. Ismo