Getting Data In

Finding and removing all but one duplicate doc

Contributor

All,

I'm curious, is there an easy way to find all duplicate logs and delete all but one of them?

Thanks!

Tags (2)
0 Karma
1 Solution

SplunkTrust
SplunkTrust

You could do something like this:

base search | streamstats count by _raw | where count > 1

That should select duplicates number 2, 3, and so on. Once you've confirmed that this really is what you're looking for, you can switch to a user with the can_delete role and pipe that to delete.

View solution in original post

SplunkTrust
SplunkTrust

You could do something like this:

base search | streamstats count by _raw | where count > 1

That should select duplicates number 2, 3, and so on. Once you've confirmed that this really is what you're looking for, you can switch to a user with the can_delete role and pipe that to delete.

View solution in original post