https://community.splunk.com/t5/Knowledge-Management/how-to-remove-duplicate-event-when-I-am-using-data-model/m-p/270110#M2373 <P>The correct answer depends on the exact nature of your data, but you might be able to get rid of duplicate events by defining a set of fields that combined ensure events are unique and then adding <EM>splitrow</EM> statements for each of the fields, e.g.</P> <PRE><CODE>| pivot datamodel object first(someField) splitrow field1 splitrow field2 splitrow field3 </CODE></PRE> Tue, 06 Jun 2017 01:21:27 GMT helge 2017-06-06T01:21:27Z https://community.splunk.com/t5/Knowledge-Management/how-to-remove-duplicate-event-when-I-am-using-data-model/m-p/270109#M2372 <P>Hi all,</P> <P>I am using data model acceleration method to do the dashboard acceleration. However, I find duplicate events in splunk. I know if use search, we can use dedup command. However, for data model root events, it doesn't allowed and "|" , for example, my original constrain is "index=123", now i want to remove those duplicate events before building data model, but splunk doesn't allow me write :"index=123 |dedup _row". Besides, from my dashboard panels, all panels created from data model must started with "|pivot data model name", and I also can't add any dedup command. Can anyone help me on this?</P> <P>Thanks a lot</P> Mon, 25 Jul 2016 02:42:08 GMT https://community.splunk.com/t5/Knowledge-Management/how-to-remove-duplicate-event-when-I-am-using-data-model/m-p/270109#M2372 hqw 2016-07-25T02:42:08Z https://community.splunk.com/t5/Knowledge-Management/how-to-remove-duplicate-event-when-I-am-using-data-model/m-p/270110#M2373 <P>The correct answer depends on the exact nature of your data, but you might be able to get rid of duplicate events by defining a set of fields that combined ensure events are unique and then adding <EM>splitrow</EM> statements for each of the fields, e.g.</P> <PRE><CODE>| pivot datamodel object first(someField) splitrow field1 splitrow field2 splitrow field3 </CODE></PRE> Tue, 06 Jun 2017 01:21:27 GMT https://community.splunk.com/t5/Knowledge-Management/how-to-remove-duplicate-event-when-I-am-using-data-model/m-p/270110#M2373 helge 2017-06-06T01:21:27Z https://community.splunk.com/t5/Knowledge-Management/how-to-remove-duplicate-event-when-I-am-using-data-model/m-p/270111#M2374 <P>I'm facing the same situation where i found duplicates in my datamodel, because the dataset I created for the model is root event based and I have duplicates in my indexed events , and I couldn't find any command to de-duplicate the data from the model, so a workaround for that is to create a dataset for the data model based on root search instead of root event, and in that search add a dedup command, that way the data in the data model should not have duplicates, but it would be easier if there was a command that be used after creating the data model to deduplicate the data, because in my case i had to recreate the dataset of my datamodel</P> Thu, 09 Jan 2020 13:51:02 GMT https://community.splunk.com/t5/Knowledge-Management/how-to-remove-duplicate-event-when-I-am-using-data-model/m-p/270111#M2374 oajengui 2020-01-09T13:51:02Z