topic Re: How do you reload a file? in Getting Data In

How do you reload a file?

jldebell — Mon, 16 Mar 2015 21:37:11 GMT

The permissions were incorrect for files being monitored. The files appeared to be indexed but they are not in Splunk. I went in and altered the file to trick the CRC Check Sum thinking it would trick the system into re-indexing the items. I get the following messages when I saved the revised files (real-time) in the _internal index:

Will begin reading at offset=0 for file=

And

group=per_source_thruput, series="/opt/splunk/*.txt", kbps=###, eps=###, kb=###, ev=###, avg_age=###, max_age=###

I am not seeing denied/failed messages. The information is still not indexing.

Please let me know if you have any suggestions.

Thanks, Jenn

Re: How do you reload a file?

esix_splunk — Mon, 28 Sep 2020 19:14:02 GMT

Quickest way is to delete a specific file from the fishbucket (state monitoring.)

./splunk cmd btprobe -d /path/to/splunk/var/lib/splunk/fishbucket/splunk_private_db --file /full/path/to/filename.txt --reset

That will reset Splunk's internal monitor for files, and force it to reread the specific file. If you have only a hand full of files, this works easily. If you're dealing with thousands of files, then you'd want to script this as wildcards do not work.

Here is a good answers article on various methods : http://answers.splunk.com/answers/72562/how-to-reindex-data-from-a-forwarder.html

Re: How do you reload a file?

glennpierce — Mon, 16 Mar 2015 22:01:35 GMT

Hi Jenn,

I think this Splunk Answer may be what you're after:

http://answers.splunk.com/answers/72562/how-to-reindex-data-from-a-forwarder.html

Re: How do you reload a file?

somesoni2 — Mon, 16 Mar 2015 22:02:33 GMT

What is your data input configuration (inputs.conf ) from the forwarder? When updating the content to trick the CRC, what portion of the file you updated, from start of file or end of file?

Re: How do you reload a file?

jldebell — Mon, 16 Mar 2015 22:49:02 GMT

I altered the start of the file. I added a space initially, but it didn't work. It would not let me save it stating there were no changes. I then added | (pipes). The file updated, but did not index.

Re: How do you reload a file?

jldebell — Mon, 16 Mar 2015 22:50:39 GMT

If I understand correctly, this will clean out the file so then I can re-index it. It won't impact other files since I am specifying which to look for. I will try this.

Re: How do you reload a file?

jldebell — Mon, 16 Mar 2015 22:52:20 GMT

Thanks for the reference. i was reading about the one shot and fish bucket clean up. I will see if this will work. Thanks!

Re: How do you reload a file?

jldebell — Mon, 28 Sep 2020 19:10:51 GMT

I tried the string, but I am getting a file path not recognized. I am in the server, drilled down to the splunk_private_db and then added the string. The first path was the splunk_private_bd and the second was the path to the file. I was able to tab and have it pull the information (auto-fill function in Unix). Which I would expect that if I tab and it auto-fills that the path exists. I am guessing it is a user error and I played around based on documentation, but I am not catching it.

In the server:
logged in as Splunk.

navigate to the /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/ folder

add the following details:

./splunk cmd btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/ --file /opt/splunk/F1/F2/text_file.TXT --reset

Error message:

-bash: ./splunk: No such file or directory

I am going to search on error messages related to the command and see if i can find anything. Please keep me posted if you see anything I missed.

Thanks, Jenn

Re: How do you reload a file?

glennpierce — Tue, 17 Mar 2015 22:42:35 GMT

If your using bash on a *nix based system make sure your in the $SPLUNK_HOME/bin directory before you run that command. Or add /opt/splunk/bin/splunk cmd [etc] to your command.

Re: How do you reload a file?

jldebell — Wed, 08 Apr 2015 21:14:00 GMT

The files were not able to reload because there was a TIME_FORMAT error associated. I was able to get the formatting corrected and reload the files. The error message was over a week old, but it was the root cause. I have included the link to the other question associated in case others encounter something similar.

http://answers.splunk.com/answers/224111/how-do-i-properly-describe-non-standard-datetime-f.html#answer-224256

Thanks again for everyone's assistance.