Hi
We have a splunk add-on for aws to pull the logs from s3 bucket. we are using the sqs based s3 inputs created to read the logs for s3 bucket, however we are noticing that through this option splunk service seems like ommiting some files from reading even though it has consumed the sqs message and deleted the message from the quer. I am attaching one of the example from our issue where in at particular time frame which is on july 6th 19 to 20 hrs we have 59 objects in the s3 bucket but splunk had read only 58 files. This is being one of the example to show but we are having this issue very often every hour one or 2 files missing. We have around 8000 to 10000 events in each file which is missing indexed in splunk due to this issue. I have checked all the internal logs which does not show any failure messages while reading this particular s3 object to confirm it was dropped or failed while parsing and processing. Its just not there. these issue is there every day every hour missing one or the other files missed by splunk inputs. From SQS perspective SQS based S3 input is skipping some objects from s3 but deleting the message from sqs
Thanks and Regards Srini
