https://community.splunk.com/t5/Getting-Data-In/Firehose-through-HTTP-Event-Collector-generates-duplicate-log/m-p/568147#M100842 <P>Hello everyone,</P><P>I am streaming CloudWatch logs to SPLUNK through Firehose, and I faced the following issue:</P><P>Some json records are being indexed(?) twice and show up twice in search. The only difference between the records is the time of indexing.<BR />I am trying to figure out how I can debug the issue. Record shows up only once in source log group in cloudwatch and s3 backups. It’s either Firehose sending a particular record twice or SPLUNK processing the same record two times. Do you have an idea how I can check my theories? I didn’t find much useful info in splunk http event collector logs. It has only technical info about the transaction: size/speed/time.</P> Wed, 22 Sep 2021 20:27:37 GMT mmkt 2021-09-22T20:27:37Z https://community.splunk.com/t5/Getting-Data-In/Firehose-through-HTTP-Event-Collector-generates-duplicate-log/m-p/568147#M100842 <P>Hello everyone,</P><P>I am streaming CloudWatch logs to SPLUNK through Firehose, and I faced the following issue:</P><P>Some json records are being indexed(?) twice and show up twice in search. The only difference between the records is the time of indexing.<BR />I am trying to figure out how I can debug the issue. Record shows up only once in source log group in cloudwatch and s3 backups. It’s either Firehose sending a particular record twice or SPLUNK processing the same record two times. Do you have an idea how I can check my theories? I didn’t find much useful info in splunk http event collector logs. It has only technical info about the transaction: size/speed/time.</P> Wed, 22 Sep 2021 20:27:37 GMT https://community.splunk.com/t5/Getting-Data-In/Firehose-through-HTTP-Event-Collector-generates-duplicate-log/m-p/568147#M100842 mmkt 2021-09-22T20:27:37Z