Finding and removing all but one duplicate doc

[bruceclarke]

All,

I'm curious, is there an easy way to find all duplicate logs and delete all but one of them?

Thanks!

[martin_mueller · Accepted solution]

You could do something like this:

base search | streamstats count by _raw | where count > 1

That should select duplicates number 2, 3, and so on. Once you've confirmed that this really is what you're looking for, you can switch to a user with the can_delete role and pipe that to delete.