Indexing the same file twice

[abhayneilam]

Hi,

I have an index called "XYZ" and in it i have a file called "abc.txt" and I am taking the help of a configuration file to import "abc.txt" in splunk automatically. Now , I have by mistaken copied it twice in the same location where my splunk pics up the data "abc.txt" for indexing. Now, I am getting double of data, plz help me to delete the data which had been indexed twice. My Index is still showing the duplicate data even after removing the duplicate file from the location.

Regards,
Abhay

[somesoni2]

Since you copied your abc.txt file to same location, it must have gotten renamed (to abc_copy.txt or something similar). Execute following command to delete the duplicate data.

index=XYZ source=abc_copy.txt | delete

The user which will execute this search should have 'can_delete' capabilities added.

[abhayneilam]

Many Many thanks for demonstrating me the whole scenario.

[lukejadamec · Accepted solution]

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

[abhayneilam]

Many thanks for the reply !!