Getting Data In

remove source and source types

Path Finder

Hi

I was testing splunk by adding multiple servers by both syslog and splunk forwarder. But now my splunk indexer has all kinds of stuff that I don't need anymore.

I want to remove some hosts, source and source types and the data it contains. Per the document I should be able to delete by removing stanzas in indexes.conf. But i can't find the entries in it.

Please help me out.

Regards,
Harish

Tags (3)
1 Solution

Ultra Champion

No,

to stop new events from coming into your indexes - remove the [monitor] and TCP/UDP stanzas in inputs.conf or go through the GUI to remove them. This, however does not remove any data already indexed.

If you delete stuff from indexes.conf, you remove the reference to where the indexes are located on disk. I don't think that's what you want.

If you selectively want to remove some of the information in the indexes you otherwise wish to keep - then you should type in a search that selects those events and pipe them to delete.

Make sure that your user role has the can_delete privilege (Manager -> Access Control -> Roles -> your_role ). Then, assuming you have data from one host (badhost) you wish to delete and data from another host (goodhost) you wish to keep:

host=badhost | delete

Whoops, now all data from badhost is gone. However, the data from goodhost is of two different sourcetypes (badtype and goodtype) - guess what:

host=goodhost sourcetype=badtype | delete

..and it's gone as well.

You should note that this operation does NOT actually delete the data, it just prevents it from ever being returned as search results.

If you want to clean an index completely, then you should do so through the command line:

  • stop splunk

  • type /opt/splunk/bin/splunk clean eventdata -index <the index you wish to clean>

  • start splunk again

For more information regarding the clean command see:
http://splunk-base.splunk.com/answers/8191/is-there-a-way-to-clean-event-data-from-a-specific-date-r...

hope this helps,

Kristian

EDIT: Missed a parameter in the clean command... oops. fixed now.

View solution in original post

Ultra Champion

No,

to stop new events from coming into your indexes - remove the [monitor] and TCP/UDP stanzas in inputs.conf or go through the GUI to remove them. This, however does not remove any data already indexed.

If you delete stuff from indexes.conf, you remove the reference to where the indexes are located on disk. I don't think that's what you want.

If you selectively want to remove some of the information in the indexes you otherwise wish to keep - then you should type in a search that selects those events and pipe them to delete.

Make sure that your user role has the can_delete privilege (Manager -> Access Control -> Roles -> your_role ). Then, assuming you have data from one host (badhost) you wish to delete and data from another host (goodhost) you wish to keep:

host=badhost | delete

Whoops, now all data from badhost is gone. However, the data from goodhost is of two different sourcetypes (badtype and goodtype) - guess what:

host=goodhost sourcetype=badtype | delete

..and it's gone as well.

You should note that this operation does NOT actually delete the data, it just prevents it from ever being returned as search results.

If you want to clean an index completely, then you should do so through the command line:

  • stop splunk

  • type /opt/splunk/bin/splunk clean eventdata -index <the index you wish to clean>

  • start splunk again

For more information regarding the clean command see:
http://splunk-base.splunk.com/answers/8191/is-there-a-way-to-clean-event-data-from-a-specific-date-r...

hope this helps,

Kristian

EDIT: Missed a parameter in the clean command... oops. fixed now.

View solution in original post

Communicator

Thanks a lot kristian.kolb !!
this answer helped me a lot ....

0 Karma

Path Finder

What success message does it gives when we delete the index or source?

0 Karma

Engager

Splunk -> Manager -> Data inputs -> (Files and Directories in my own case) -> locate your data source and on the right side under "Actions", click delete.

Hope this helps

Contributor

what i notice in v4.3.1 for linux is that source types are created in weird ways. if you create a new source (file) and you dont preview the data and you use manual source name, the source is not added to the props.conf file and will never be seen in the "list" when creating a new source, but, if you use the same name again it will complain with a "that name already exists" error message. but if you preview the data and select "create new source" it will populate props.conf with the new source type and will be available in the "list". but no place to edit/delete sources from the GUI?

Contributor

Certain types of sources do not appear in this list.

0 Karma