How to deal with data that was uploaded twice?

[sudarshan391]

Hi Experts,

I am now in a strange situation, we have a index in which we uploaded .csv files for every month and for previous month data has been uploaded two times. now splunk is showing duplicate entries.

Can someone please suggest how can I get through this situation?
I want to remove duplicate entries for last month from index.

Thanks.

Regards,
Sud

[somesoni2]

You can use delete command to remove those duplicate records from any future search (it actually makes those records unsearchable but doesn't actually deletes it/removes from disk). See this for more information on the same.
https://docs.splunk.com/Documentation/Splunk/7.0.0/Indexer/RemovedatafromSplunk#Delete_events_from_s...

Please ensure that you run the search without delete command first and validate that you got only the records that you want to delete.

[prosenjit2707]

Check out this link:
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Delete