Splunk Enterprise

Timestamp extraction is not working

iamvinaykumar
Engager

WE have data coming from syslog which is like below :

2021-06-16T19:03:02+02:00 XXXXXXXXXX  - (6/16/21 5:03:02.000 PM - splunk extracted time)

2021-06-16T19:02:58+02:00 XXXXXXXXXX - (6/16/21 5:02:58.000 PM  - splunk extracted time)

these data's are sent to indexer from syslog via heavy forwarders , data from syslog is actually in the UTC time zone but +2:00 have been appended wrongly in the syslog data , splunk intelligence is taking +2:00 and substracting it from utc time and there is 2 hour time gap , 

inorder to extract correct time stamp we placed props.conf in the heavy forwarder (config below) but still the extraction is not working . ensured that on disk config is coming as below but still required timestamps are not extracted , still there is time gap of 2 hours .

 

TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %Y-%m-%dT%H%M%S
LINE_BREAKER = ([\n\r]+)
SHOULD_LINEMERGE = false
TRUNCATE = 999999

Any solutions to extract highlighted time ?

 

Labels (2)
0 Karma

isoutamo
SplunkTrust
SplunkTrust
Have you try TZ=UTC on props.conf? I suppose that HFs own time zone is what it is using now even you cut of that +02:00 from event.
Another option is try use INGEST_EVAL = _time := .... setting first time with correct TZ and then do strptime for it when assign it to _time.
r. Ismo
0 Karma

iamvinaykumar
Engager

@isoutamo  Yes we tried to set TZ = UTC but still  Splunk is reading from the log , Also tried to set the time zone in  source i e universal forwarder forwarder in the syslog servers . but even that didn't helped . 

Will index-time eval function work in this case ?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Here is one example to use INGEST_EVAL for manipulating time https://www.tekstream.com/a-use-case-for-ingest-time-eval/

r. Ismo

Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...