Getting Data In

Indexing the same file twice

abhayneilam
Contributor

Hi,

I have an index called "XYZ" and in it i have a file called "abc.txt" and I am taking the help of a configuration file to import "abc.txt" in splunk automatically. Now , I have by mistaken copied it twice in the same location where my splunk pics up the data "abc.txt" for indexing. Now, I am getting double of data, plz help me to delete the data which had been indexed twice. My Index is still showing the duplicate data even after removing the duplicate file from the location.

Regards,
Abhay

Tags (2)
0 Karma
1 Solution

lukejadamec
Super Champion

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

View solution in original post

somesoni2
Revered Legend

Since you copied your abc.txt file to same location, it must have gotten renamed (to abc_copy.txt or something similar). Execute following command to delete the duplicate data.

index=XYZ source=abc_copy.txt | delete

The user which will execute this search should have 'can_delete' capabilities added.

0 Karma

abhayneilam
Contributor

Many Many thanks for demonstrating me the whole scenario.

0 Karma

lukejadamec
Super Champion

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

abhayneilam
Contributor

Many thanks for the reply !!

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...