Splunk Enterprise

Timestamp extraction is not working

iamvinaykumar
Engager

WE have data coming from syslog which is like below :

2021-06-16T19:03:02+02:00 XXXXXXXXXX  - (6/16/21 5:03:02.000 PM - splunk extracted time)

2021-06-16T19:02:58+02:00 XXXXXXXXXX - (6/16/21 5:02:58.000 PM  - splunk extracted time)

these data's are sent to indexer from syslog via heavy forwarders , data from syslog is actually in the UTC time zone but +2:00 have been appended wrongly in the syslog data , splunk intelligence is taking +2:00 and substracting it from utc time and there is 2 hour time gap , 

inorder to extract correct time stamp we placed props.conf in the heavy forwarder (config below) but still the extraction is not working . ensured that on disk config is coming as below but still required timestamps are not extracted , still there is time gap of 2 hours .

 

TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %Y-%m-%dT%H%M%S
LINE_BREAKER = ([\n\r]+)
SHOULD_LINEMERGE = false
TRUNCATE = 999999

Any solutions to extract highlighted time ?

 

Labels (2)
0 Karma

isoutamo
SplunkTrust
SplunkTrust
Have you try TZ=UTC on props.conf? I suppose that HFs own time zone is what it is using now even you cut of that +02:00 from event.
Another option is try use INGEST_EVAL = _time := .... setting first time with correct TZ and then do strptime for it when assign it to _time.
r. Ismo
0 Karma

iamvinaykumar
Engager

@isoutamo  Yes we tried to set TZ = UTC but still  Splunk is reading from the log , Also tried to set the time zone in  source i e universal forwarder forwarder in the syslog servers . but even that didn't helped . 

Will index-time eval function work in this case ?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Here is one example to use INGEST_EVAL for manipulating time https://www.tekstream.com/a-use-case-for-ingest-time-eval/

r. Ismo

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...