All Apps and Add-ons

Kinesis AWS Input - Timestamp recognition

606866581
Path Finder

I use kinesis heavily to pull a wide variety of logs from AWS. However I've noticed the timestamps within the logs are being ignored. Some of the events don't have timestamps and so the automatic _time is acceptable, however when my log starts with
2018-07-17T16:10:00Z
I'd expect _time to be the same (adusted for timezone), however the AWS Add-on set _time to be 16:11:56 which appears nowhere in the log.

So it appears kinesis has some internal logic for calculating the timestamp and ignores the timestamp at the start of the log.

Has anyone else had this issue or found a solution?

0 Karma

snosurfur
Engager

We also use kinesis, and are pulling in vpcflow logs and recently found a solution to the log time. I will share my thoughts in hopes that it either turns out to be a solution for you or point you in the right direction.

Option 1:
This will use the current time of the splunk server to log the time of the event and convert/read epoch time.

Edit/Create a props.conf file in %SPLUNK%\etc\system\local

[]
TIME_PREFIX = CURRENT

TIME_FORMAT = %s

Option 2:
This will take either the start time of the event or end time, however you configure regexp.

Edit/Create a props.conf file in %SPLUNK%\etc\system\local

Start time of event
[]
TIME_PREFIX = ^([^ ]+\s){10}

TIME_FORMAT = %s

End time of event
[]
TIME_PREFIX = ^([^ ]+\s){11}

TIME_FORMAT = %s

Keeping in mind these are configured specifically formatted for vpc flow logs.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...