How is the data getting into Splunk? If it was coming via a forwarder, the forwarder should have automatically noticed when the indexer was down and held the data until it was back up.
If you have a single Splunk instance handling the input handling from start to finish, you may want to check out the followTail
setting for its stanza in inputs.conf
:
http://docs.splunk.com/Documentation/Splunk/latest/admin/inputsconf
It may not work in your use case, but in situations like this I often find it's easier to just clear the index, as well as the "_thefishbucket
" index which is used to keep track of data that has been indexed:
splunk stop
splunk clean eventdata -index main #or whatever your index is
splunk clean eventdata -index _thefishbucket
splunk start
Beware, though, this will cause all of your inputs to be re-indexed, and remove all data from the the main index, or whichever other index you specify.
Alternatively, if you're missing data from some entire log files, you could use the CLI:
http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorfilesanddirectoriesusingtheCLI
splunk add oneshot
can be used to add a single file. Make sure you specify the index, host and sourcetype if necessary.
Yes, you can use the splunk
binary on the forwarder for this. Make sure you specify the sourcetype, index, and anything else that would normally be set in inputs.conf
.
I am using a forwarder. Can I use add oneshot
to add a file from another server?
"the forwarder should have automatically noticed when the indexer was down and held the data until it was back up." - If Splunk was down for a few weeks then the local queue would have filled up and the forwarder would have begun to drop events, leading to the missing data.