Hi,
I am using Splunk AWS add-on to ingest AWS Kinesis stream input to Splunk Enterprise. Input works fine and i am able to get real-time data into Splunk for various CloudWatch log sources via single Kinesis stream.
Recently, i faced some issue and Splunk Heavy Forwarder was down and when i restored the services, kinesis start to ingest the data again. I noticed that during the shutdown, kinesis stream data is missing in Splunk. Kinesis only streams the latest data into splunk while any older missing data is not ingested.
Then, i changes the kinesis input configuration to TRIM_HORIZON and replicated the scenario, it resolved the missing data problem but also resulted in Duplicate data for 24 hours (Kinesis data retention period minimum confiugration).
Have you guys faced such scenario before or if there is any such configuration where i can set the datetime for Kinesis input to ingest data into Splunk.
My end goal is to achieve the high availability of data.
Regards,
Jatin
