Hi all,
I am using data model acceleration method to do the dashboard acceleration. However, I find duplicate events in splunk. I know if use search, we can use dedup command. However, for data model root events, it doesn't allowed and "|" , for example, my original constrain is "index=123", now i want to remove those duplicate events before building data model, but splunk doesn't allow me write :"index=123 |dedup _row". Besides, from my dashboard panels, all panels created from data model must started with "|pivot data model name", and I also can't add any dedup command. Can anyone help me on this?
Thanks a lot
I'm facing the same situation where i found duplicates in my datamodel, because the dataset I created for the model is root event based and I have duplicates in my indexed events , and I couldn't find any command to de-duplicate the data from the model, so a workaround for that is to create a dataset for the data model based on root search instead of root event, and in that search add a dedup command, that way the data in the data model should not have duplicates, but it would be easier if there was a command that be used after creating the data model to deduplicate the data, because in my case i had to recreate the dataset of my datamodel
The correct answer depends on the exact nature of your data, but you might be able to get rid of duplicate events by defining a set of fields that combined ensure events are unique and then adding splitrow statements for each of the fields, e.g.
| pivot datamodel object
first(someField)
splitrow field1
splitrow field2
splitrow field3