Getting Data In

How to delete logs in a tsidx index?

gcusello
SplunkTrust
SplunkTrust

Hi at all,

I installed Splunk App for BlueCoat.
I loaded some test data and now I have to delete them before loading the production data.
Logs are loaded in the "bcoat_logs" index and I have no problems to delete them.
Instead, the problem is that App loaded logs also in a tsidx index and I don't know how to delete them.

Can anyone help me?

Thank you.
Giuseppe

0 Karma
1 Solution

renjith_nair
SplunkTrust
SplunkTrust

Hello @cusello ,

I presume that you are talking about the tsidx files located in db folder.

tsidx files are index files. A tsidx file associates each unique keyword in your data with location references to events, which are stored in a companion rawdata file. Together, the rawdata file and its related tsidx files make up the contents of an index bucket.

If you do not need the data on bcoat_logs index, then you can use below command to delete the data. Please be careful, it removes entire data and you need to shutdown indexer to remove data

./splunk clean eventdata -index bcoat_logs

By any chance, if you have an index name with tsidx, then you can not delete specific events from the index. Instead you can use splunk delete command to "hide" the data

Happy Splunking!

View solution in original post

ngatchasandra
Builder

Hi cusello,

tsidx is an file or index?

If you want to delete this logs, try to run a search who will displays you your logs and delete them after like follow link:

http://docs.splunk.com/Documentation/Splunk/4.1/Admin/RemovedatafromSplunk

use the delete operator, run a search that returns the events you want deleted. Make sure that this search returns ONLY events you want to delete, and no other events.

For example, if you want to remove the events you've indexed from a source called /fflanda/incoming/cheese.log so that they no longer appear in searches, do the following:

  1. Disable or remove that source so that it no longer gets indexed.

  2. Search for events from that source in your index:

    source="/fflanda/incoming/cheese.log"

  3. Look at the results to confirm that this is the data you want to delete.

  4. Once you've confirmed that this is the data you want to delete, pipe the search to delete:

    source="/fflanda/incoming/cheese.log" | delete

0 Karma

gcusello
SplunkTrust
SplunkTrust

The Splunk App for BlueCoat stores logs in the bcoat_logs index and also in a tsidx called "bluecoat_stats " as you can see in the "BlueCoat - Stats - Collect" search:
bcoat_request | table _time action bytes_in bytes_out category cs_uri_path cs_uri_scheme dest_host filter_result http_content_type http_referrer http_user_agent src_ip src_user x_bluecoat_application_name x_virus_id | tscollect namespace=bluecoat_stats

I already deleted logs from the bcoat_logs index, but the problem is that I don't know how to delete them from the "bluecoat_stats", because the delete command cannot be executed on a "|tstats" command, in the Summary index there aren't logs from Splunk App for BlueCoat and there isn't an index with this name to clear with the CLI command.

when documentation speaks about tsidxstats "This must be manually managed if tscollect is used to create the files." I don't know where is this information "to manually manage tsidxstats" and in the tscollect documentation there isn't any useful information.

Thank you.
Giuseppe

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Hello @cusello ,

I presume that you are talking about the tsidx files located in db folder.

tsidx files are index files. A tsidx file associates each unique keyword in your data with location references to events, which are stored in a companion rawdata file. Together, the rawdata file and its related tsidx files make up the contents of an index bucket.

If you do not need the data on bcoat_logs index, then you can use below command to delete the data. Please be careful, it removes entire data and you need to shutdown indexer to remove data

./splunk clean eventdata -index bcoat_logs

By any chance, if you have an index name with tsidx, then you can not delete specific events from the index. Instead you can use splunk delete command to "hide" the data

Happy Splunking!

gcusello
SplunkTrust
SplunkTrust

I already deleted data from the bcoat_logs index but data is still in the tsidx file and I don't know how to delete them.
Thank you.
Giuseppe

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

If you have deleted the data from bcoat_logs using ./splunk clean eventdata -index bcoat_logs, then you don't need to worry about the idx files and it's not advisable to remove those files manually.

Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...