Splunk Search

Rebuilding index level .data files

vbumgarner
Contributor

On a healthy index, these two queries return the same value, or at least very similar, since the value is changing as data is indexed:

 |metadata type=sourcetypes | stats sum(totalCount)
 |dbinspect | stats sum(eventCount)

metadata seems to use the files at

*/db/*.data

dbinspect seems to use the files one level down at

*/db/*/*.data

I believe the rebuild command can be used to rebuild the .data on a bucket by bucket basis. Is there a similar command for rebuilding the .data files at the index level, the .data files just inside db?

Tags (3)
0 Karma

Simeon
Splunk Employee
Splunk Employee

This is NOT supported, but should work...

  1. Create a "meta.dirty" file in the root directory of the index you want to rebuild.
  2. Restart splunk.
0 Karma

vbumgarner
Contributor

An answer I was given off-board was to move the *.data files at the index level aside and restart. This seems to rebuild those files from the *.data files in the buckets themselves.

It would be nice to have a simple way to rebuild all counts, in all buckets and at the index level.

0 Karma

DUThibault
Contributor

The "root directory of the index" is e.g. $SPLUNK_DB/defaultdb/db/ ($SPLUNK_DB/defaultdb/ will NOT work). With Splunk 7, meta.dirty is deleted from db/ upon restart but the index is not rebuilt.

I found the following method on https://answers.splunk.com/answers/72562/how-to-reindex-data-from-a-forwarder.html (dating back to 2013):
1) # splunk stop
2) # splunk clean eventdata -index main
This sort of worked, except older data did not get re-indexed. My horizon shrunk from several days to about 5 hours. It ended up easier to remove the data sources (which were directories under surveillance anyway) and add them back in.

0 Karma
Get Updates on the Splunk Community!

How to send events & findings from AWS to Splunk using Amazon EventBridge

Amazon EventBridge is a serverless service that uses events to connect application components together, making ...

Exciting News: The AppDynamics Community Joins Splunk!

Hello Splunkers,   I’d like to introduce myself—I’m Ryan, the former AppDynamics Community Manager, and I’m ...

The All New Performance Insights for Splunk

Splunk gives you amazing tools to analyze system data and make business-critical decisions, react to issues, ...