<your search for relevant events> | eval eid=_cd | search [<same search for relevant events> | streamstats count by _raw | search count>1 | eval eid=_cd | fields eid] | delete
There is no log file to delete … your data has been indexed. To get rid of duplicate data from index… you can run a search to identify the duplicate events and pipe it to
You can use the delete command. https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Delete
If you can write a search that limits the data to one copy, then you can pipe it to the delete command so you won't see both.
This will only stop you from seeing it, it will not get rid of the data off the indexer.
You can also clean indexes if one of the copies of data is in a different index than the other you can wipe one of the indexes of all the data in there. http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/RemovedatafromSplunk#Remove_data_from_one_...
Your Search That Shows Duplicates AND NOTHING ELSE | dedup _raw | delete
You will probably have to extend your user's role/permissions to run the