Knowledge Management

If a file is ignored due to IgnoreOlderThan, is there any way to configure Splunk to read it if I need it later?

Flynt
Splunk Employee
Splunk Employee

If I specify a value for ignoreOlderThan for an input in my inputs.conf and tailingprocessor thinks the file is older than the ignoreOlderThan value and therefore ignores the file,
is there any way for the file to be read again by Splunk?

1 Solution

bmacias84
Champion

The answer is yes, but of course this depends on your requirements. The simiplist answer is to use oneshot using the CLI. You can issue one shot remotly on any splunk instance so long as you have enable remote access or have change the default password for the admin account.

I have on occasion scripted a remote oneshot to index files for one off cases.

If your looking to reset your fishbucket btprob is the real answer.

To reset one file in your fishbucket from your UF:

./btprobe -d SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/access.log --reset

Note: Your splunk daemon or service must be stopped and if you try to use the base directory you will recieve "could not open '/var/log/' (Access is denied.)". If you doing this on a UF the btprobe will need to be copied to your UF.

You should recieve the following message: key=0x7b100f52c71e67f1 scrc=0x6e16f6c553b73581 sptr=8070178 fcrc=0xcdcd2890474a56fb flen=0 mdtm=1347001048 wrtm=1352329761**95Record (key 0x7b100f52c71e67f1) reset.

If in my case I have hundreds of test file sitting on a UF in mulitple directories, so I wrote a perl script to feed each file in each directory to the btprobe utility.

Hope this helps you or gets you started. Dont forget to accept or thumbs up answers.

Additional Reading:

Hope this help or gets you started.

View solution in original post

bmacias84
Champion

The answer is yes, but of course this depends on your requirements. The simiplist answer is to use oneshot using the CLI. You can issue one shot remotly on any splunk instance so long as you have enable remote access or have change the default password for the admin account.

I have on occasion scripted a remote oneshot to index files for one off cases.

If your looking to reset your fishbucket btprob is the real answer.

To reset one file in your fishbucket from your UF:

./btprobe -d SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/access.log --reset

Note: Your splunk daemon or service must be stopped and if you try to use the base directory you will recieve "could not open '/var/log/' (Access is denied.)". If you doing this on a UF the btprobe will need to be copied to your UF.

You should recieve the following message: key=0x7b100f52c71e67f1 scrc=0x6e16f6c553b73581 sptr=8070178 fcrc=0xcdcd2890474a56fb flen=0 mdtm=1347001048 wrtm=1352329761**95Record (key 0x7b100f52c71e67f1) reset.

If in my case I have hundreds of test file sitting on a UF in mulitple directories, so I wrote a perl script to feed each file in each directory to the btprobe utility.

Hope this helps you or gets you started. Dont forget to accept or thumbs up answers.

Additional Reading:

Hope this help or gets you started.

DerekB
Splunk Employee
Splunk Employee

Tailing processor looks at the modtime of the file, so if you update the modtime, the file should be looked at again by Splunk. One easy way to do this is to simply restart the machine. The modtime gets updated during that process and tailing processor will then re-read the file and index any new information.

0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...