Good morning fellow Splunkthiasts!
I have an index with 100k+ events per minute (all of them having the same sourcetype), approximately 100 fields are known in this dataset. Some of these events are duplicit, while others are unique. My aim is to understand the duplication and be able to explain what events exactly get duplicated.
I am detecting duplicities using this SPL:
index="myindex" sourcetype="mysourcetype" | eventstats count AS duplicates BY _time, _raw
Now I need to identify what fields or their combination make the difference, under what circumstances the event is ingested twice.
I tried to use predict command, however it is somehow producing new values for "duplicates" field, but it does not disclose the rule by which it makes the decision. In other words, I am not interested in prediction itself, I want to know the predictors.
Is something like that possible in SPL?
