topic Re: Indexing the same file twice in Getting Data In

Indexing the same file twice

abhayneilam — Mon, 18 Nov 2013 17:21:18 GMT

Hi,

I have an index called "XYZ" and in it i have a file called "abc.txt" and I am taking the help of a configuration file to import "abc.txt" in splunk automatically. Now , I have by mistaken copied it twice in the same location where my splunk pics up the data "abc.txt" for indexing. Now, I am getting double of data, plz help me to delete the data which had been indexed twice. My Index is still showing the duplicate data even after removing the duplicate file from the location.

Regards,
Abhay

Re: Indexing the same file twice

lukejadamec — Mon, 18 Nov 2013 17:28:30 GMT

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

Re: Indexing the same file twice

somesoni2 — Mon, 18 Nov 2013 18:04:22 GMT

Since you copied your abc.txt file to same location, it must have gotten renamed (to abc_copy.txt or something similar). Execute following command to delete the duplicate data.

index=XYZ source=abc_copy.txt | delete

The user which will execute this search should have 'can_delete' capabilities added.

Re: Indexing the same file twice

abhayneilam — Mon, 18 Nov 2013 18:06:59 GMT

Many thanks for the reply !!

Re: Indexing the same file twice

abhayneilam — Mon, 18 Nov 2013 18:07:42 GMT

Many Many thanks for demonstrating me the whole scenario.