Getting Data In

Is there a way to reindex the data from 2/1 to today without reindexing the other files?

tkwaller_2
Communicator

Hello
So I have some data for some reason that did not get index in my monitored filepath. I have a feeling it has something to do with the service writing to the file. It stopped writing for some time and then we restarted it and I updated the monitor to go to a new sourcetype in the same index.
It seems the data from 1/30 to today isn't indexed, maybe since the crc didn't change?
Is there a way to reindex the data from 2/1 to today without reindexing the other files?(Ex 1/30 e4ct.)

CAn I change the crc and add ignoreOlderThan?

Thanks for the help

0 Karma
1 Solution

micahkemp
Champion

For this single use case, you can use crcSalt = <SOURCE> in inputs.conf to reindex only those files you believe need to be reindexed. You could add an entry in inputs.conf like:

[monitor://tmpdirectory]
crcSalt = <SOURCE>
index = whatever
sourcetype = whatever

Set tmpdirectory to a location that you will copy the previously unindexed files into. crcSalt = <SOURCE> in inputs.conf will tell splunk that the filename is what makes these files unique, and therefore it will index them even though it may think it has seen them before.

The above is a one time workaround, it is not intended to be used for your normal log rotation monitoring.

View solution in original post

0 Karma

micahkemp
Champion

For this single use case, you can use crcSalt = <SOURCE> in inputs.conf to reindex only those files you believe need to be reindexed. You could add an entry in inputs.conf like:

[monitor://tmpdirectory]
crcSalt = <SOURCE>
index = whatever
sourcetype = whatever

Set tmpdirectory to a location that you will copy the previously unindexed files into. crcSalt = <SOURCE> in inputs.conf will tell splunk that the filename is what makes these files unique, and therefore it will index them even though it may think it has seen them before.

The above is a one time workaround, it is not intended to be used for your normal log rotation monitoring.

0 Karma

tkwaller_2
Communicator

Hello

Good answer, however the root cause was me not configuring timestamp recognition correctly.
It seems that the field we were using for timestamp doesn't change, even when a event is updated we were still looking at one date instead of a different one, so the data from those files were there just under a different date.

Thanks again
Todd

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...