topic Re: How to delete logs in a tsidx index? in Getting Data In

How to delete logs in a tsidx index?

gcusello — Thu, 18 Feb 2016 10:58:29 GMT

Hi at all,

I installed Splunk App for BlueCoat.
I loaded some test data and now I have to delete them before loading the production data.
Logs are loaded in the "bcoat_logs" index and I have no problems to delete them.
Instead, the problem is that App loaded logs also in a tsidx index and I don't know how to delete them.

Can anyone help me?

Thank you.
Giuseppe

Re: How to delete logs in a tsidx index?

renjith_nair — Thu, 18 Feb 2016 11:31:15 GMT

Hello @cusello ,

I presume that you are talking about the tsidx files located in db folder.

tsidx files are index files. A tsidx file associates each unique keyword in your data with location references to events, which are stored in a companion rawdata file. Together, the rawdata file and its related tsidx files make up the contents of an index bucket.

If you do not need the data on bcoat_logs index, then you can use below command to delete the data. Please be careful, it removes entire data and you need to shutdown indexer to remove data

./splunk clean eventdata -index bcoat_logs

By any chance, if you have an index name with tsidx, then you can not delete specific events from the index. Instead you can use splunk delete command to "hide" the data

Re: How to delete logs in a tsidx index?

gcusello — Thu, 18 Feb 2016 11:38:16 GMT

I already deleted data from the bcoat_logs index but data is still in the tsidx file and I don't know how to delete them.
Thank you.
Giuseppe

Re: How to delete logs in a tsidx index?

ngatchasandra — Thu, 18 Feb 2016 11:59:15 GMT

Hi cusello,

tsidx is an file or index?

If you want to delete this logs, try to run a search who will displays you your logs and delete them after like follow link:

http://docs.splunk.com/Documentation/Splunk/4.1/Admin/RemovedatafromSplunk

use the delete operator, run a search that returns the events you want deleted. Make sure that this search returns ONLY events you want to delete, and no other events.

For example, if you want to remove the events you've indexed from a source called /fflanda/incoming/cheese.log so that they no longer appear in searches, do the following:

Disable or remove that source so that it no longer gets indexed.
Search for events from that source in your index:

source="/fflanda/incoming/cheese.log"
Look at the results to confirm that this is the data you want to delete.
Once you've confirmed that this is the data you want to delete, pipe the search to delete:

source="/fflanda/incoming/cheese.log" | delete

Re: How to delete logs in a tsidx index?

renjith_nair — Tue, 29 Sep 2020 08:46:44 GMT

If you have deleted the data from bcoat_logs using ./splunk clean eventdata -index bcoat_logs, then you don't need to worry about the idx files and it's not advisable to remove those files manually.

Re: How to delete logs in a tsidx index?

gcusello — Tue, 29 Sep 2020 08:50:06 GMT

The Splunk App for BlueCoat stores logs in the bcoat_logs index and also in a tsidx called "bluecoat_stats " as you can see in the "BlueCoat - Stats - Collect" search:
bcoat_request | table _time action bytes_in bytes_out category cs_uri_path cs_uri_scheme dest_host filter_result http_content_type http_referrer http_user_agent src_ip src_user x_bluecoat_application_name x_virus_id | tscollect namespace=bluecoat_stats

I already deleted logs from the bcoat_logs index, but the problem is that I don't know how to delete them from the "bluecoat_stats", because the delete command cannot be executed on a "|tstats" command, in the Summary index there aren't logs from Splunk App for BlueCoat and there isn't an index with this name to clear with the CLI command.

when documentation speaks about tsidxstats "This must be manually managed if tscollect is used to create the files." I don't know where is this information "to manually manage tsidxstats" and in the tscollect documentation there isn't any useful information.

Thank you.
Giuseppe