Splunk Search

Can we delete the fishbucket for a specific index ?

vrmandadi
Builder

Hello Experts ,

I am trying to delete the fishbucket but I want to delete only one index=syslog..Is there a command I can run that only delete for a  particular index

 

Thanks in Advance 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The fishbucket is used for Splunk to keep track of its place in each input file.  This is before data is indexed so fishbuckets have no knowledge of indexes.  Deleting a fishbucket causes an input file to re-indexed from the beginning.

If you want to delete data from an index then give up now.  Indexed data cannot be deleted, removed, purged, edited, redacted, modified, or otherwise changed.  The best you can do is hide events from search results using the delete command.

---
If this reply helps you, Karma would be appreciated.
0 Karma

vrmandadi
Builder

@richgalloway  Thank you for your response .. The reason I asked was I am having issue with data -re indexing .For that I have done the following steps
Created a new index(previously it was syslog ..changed to syslog1)
Created new data input ([monitor:///admin/logs/abc/syslog/syslog.log*]
Reset the fishbucket entry for all those files
After I enabled the input I see data coming in from syslog.log, syslog.log.25.gz, syslog.log.26.gz etc but few are missing

I checked splunkd.log and saw these messages

12-08-2022 01:50:55.675 +0000 INFO  ArchiveProcessor [180967 archivereader] - Handling file=/admin/logs/abc/syslog/syslog.log.2.gz

12-08-2022 01:50:55.676 +0000 INFO  ArchiveProcessor [180967 archivereader] - record time older than bucket, reindexing path=/admin/logs/abc/syslog/syslog.log.2.gz

12-08-2022 01:50:55.676 +0000 INFO  ArchiveProcessor [180967 archivereader] - reading path=/admin/logs/abc/syslog/syslog.log.2.gz (seek=0 len=579119)

12-08-2022 01:50:55.788 +0000 INFO  ArchiveProcessor [180967 archivereader] - Archive with path="/admin/logs/abc/syslog/syslog.log.2.gz" was already indexed as a non-archive, skipping.

12-08-2022 01:50:55.790 +0000 INFO  ArchiveProcessor [180967 archivereader] - Finished processing file '/admin/logs/abc/syslog/syslog.log.2.gz', removing from stats

 

How to re-ingest 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I think what is happening is Splunk is refusing to ingest the gzip file because it thinks it's already read the uncompressed version of the file.  If the .gz file is just a compressed version of a file already read then you're done (compressed files tend to be deny-listed to avoid this).

If you need the gzip files indexed then try this.  Denylist the .gz files and allow the rest to be indexed.  Clear the fishbucket again then denylist the uncompressed files.  This should allow the compressed files to be indexed.  After that, restore your normal input settings.

---
If this reply helps you, Karma would be appreciated.
0 Karma

vrmandadi
Builder

@richgalloway  Thank you for your input . So are you suggesting to blacklist gzip files first so that it indexes unzipped files and then blacklist unzipped files so that zip files will be indexed?.

monitor:///admin/logs/bac/syslog/syslog.log*]

blacklist = .*/syslog\.log\.1\.gz$

disabled = 0

host = metrics-preos02

host_segment = 3

index = syslog-test1

sourcetype = syslog

whitelist = .*/syslog\.log(|\.[0-9]+\.gz)$

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, that is what I am suggesting - with a delete of the fishbucket in between.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...