How to forward old data from a forwarder to a new Splunk index?

[LuiesCui]

Hey guys, I'm new to splunk and I really need ur help!!!

As what I know, once the data from a .log file are loaded by forwarder to Splunk, the forwarder won't load them again if the .log file has not been change. What I want to do is to get the old data to a new Splunk index without changing the .log file and what happens is the index gets nothing. So is there any way to clear the record (or change some configuration? i don't know) on what the forwarder has already sent and make it forward that old data again? Thx so much!!!

[bohrasaurabh]

Another approach would be backup the c:\program files\spunkforwarder\etc\apps directory on the forwarder machine and then un-install and reinstall the forwarder. place back the contents of the apps directory and then start the forwarder.

[LuiesCui]

Can I just delete the fishbucket?

[MuS]

This is some kind of a hardcore way, but it will work as well if you stop the forwarder and delete C:\Program Files\SplunkUniversalForwarder\var\lib\splunk\fishbucket the directory will then be re-created if you start the forwarder again.

[MuS · Accepted solution]

Hi LuiesCui,

you can either clean the index fishbucket on the forwarder by running the following command:

$SPLUNK_HOME/bin/splunk clean all

This will force the forwarder the re-load everything or you add the following option crcSalt = REINDEXMEPLEASE to your monitor stanza in inputs.conf:

[monitor://C:\temp\tutorialdata]
disabled = 0
index = foo
sourcetype = myfoo
crcSalt = REINDEXMEPLEASE

restart the forwarder, wait some seconds and immediately remove the option from inputs.conf without restarting the forwarder.
This will enable you to selectively re-load single inputs. Both methods tested and working on Windows with Splunk universal forwarder 6.1.1

Hope that helps ...

cheers, MuS

[dantimola]

Hi, MuS,

What will happen if he didn't remove the option from inputs.conf?

Thanks,
Dan

[MuS]

Hi dantimola,

if you don't remove it, the universal forwarder will re-index all the things all the time for this monitor. By adding it once and removing it immediately the uf will just re-index all the things once.

Hope this answers your question?

cheers, MuS

[LuiesCui]

Thank u so much! Still I have some questions about the fishbucket. I'm using the forwarder on win 7 and I think where I type the command is the cmd.exe right? But what cmd.exe shows is "is not recognized as an internal or external command,
operable program or batch file." And the way by editing input.conf works once. If I do the same thing on the same .log files again, no data loaded.

[MuS]

On windows it's probably c:\program files\spunkforwarder\bin\splunk clean eventdata -index _thefishbucket . The inputs.conf method will work each time, don't forget to restart the forwarder after the modification.

[LuiesCui]

Well the input.conf method is still not working. And the cli method shows "Error: Cleaning eventdata is not supported on this version." I am using the forwarder of 6.1.1 32bit on win 7 64bit. What would happen if I just delete the fishbucket?

[MuS]

my bad, sorry the clean command was from an indexer ..... see my updates in the answer, both will work for you.

[LuiesCui]

Thank you so much!