I can see that we are having duplicate events in every index, query used to identify the duplicate events:
index=* |eval myID=_cd |search [search index=* |streamstats count by _raw |search count>1|eval myID=_cd |fields myID ] |stats c(myID) as dpc by index
Query used to get bucket details of these events:
index=* | eval cd=_cd | eval bkt= _bkt | table cd bkt index splunk_server _time source host sourcetype _raw
Note: SF and RF are not met and are set to 3:3. We have multisite clustered environment.
Could this issue be due to SF RF not met or somehow SH is showing up data from replicated buckets as well? Is there a fix to this?