Splunk Enterprise

Timestamp extraction is not working

iamvinaykumar
Engager

WE have data coming from syslog which is like below :

2021-06-16T19:03:02+02:00 XXXXXXXXXX  - (6/16/21 5:03:02.000 PM - splunk extracted time)

2021-06-16T19:02:58+02:00 XXXXXXXXXX - (6/16/21 5:02:58.000 PM  - splunk extracted time)

these data's are sent to indexer from syslog via heavy forwarders , data from syslog is actually in the UTC time zone but +2:00 have been appended wrongly in the syslog data , splunk intelligence is taking +2:00 and substracting it from utc time and there is 2 hour time gap , 

inorder to extract correct time stamp we placed props.conf in the heavy forwarder (config below) but still the extraction is not working . ensured that on disk config is coming as below but still required timestamps are not extracted , still there is time gap of 2 hours .

 

TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %Y-%m-%dT%H%M%S
LINE_BREAKER = ([\n\r]+)
SHOULD_LINEMERGE = false
TRUNCATE = 999999

Any solutions to extract highlighted time ?

 

Labels (2)
0 Karma

isoutamo
SplunkTrust
SplunkTrust
Have you try TZ=UTC on props.conf? I suppose that HFs own time zone is what it is using now even you cut of that +02:00 from event.
Another option is try use INGEST_EVAL = _time := .... setting first time with correct TZ and then do strptime for it when assign it to _time.
r. Ismo
0 Karma

iamvinaykumar
Engager

@isoutamo  Yes we tried to set TZ = UTC but still  Splunk is reading from the log , Also tried to set the time zone in  source i e universal forwarder forwarder in the syslog servers . but even that didn't helped . 

Will index-time eval function work in this case ?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Here is one example to use INGEST_EVAL for manipulating time https://www.tekstream.com/a-use-case-for-ingest-time-eval/

r. Ismo

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.