I setup a Universal Forwarder forwarding some CSV files to three indexers. I made the mistake of forwarding the data before setting the indexes on the indexers. So now the status is this: The forwarder shows that all data files are forwarded; I see one message like the one below in each indexer's splunkd.log:
02-26-2013 09:47:15.697 -0500 WARN IndexProcessor - received event for unconfigured/disabled index='dcmon' with source='source::/opt/var/log/data-2013-02-21.csv' host='host::fwdr-prod01' sourcetype='sourcetype::dcmon' (1 missing total)
After the indexes are setup, I did a "clean all" on the forwarder. But I still am not able to find any event for this data. The daily CSV file grows every 15 minutes and the forwarder continues to show that new data is forwarded. But the indexes on the indexers are still zero in size.
By the way, other data from the same forwarder can be found on the indexers.
Any pointers are greatly appreciated.
[edit] The indexes ("dcmon") on all three indexers are showing "Enabled" under status.
You will have to clean the fishbucket on the indexer as well.
Did cleaning the fishbucket correct your problem?
Correct on the UF. Keep in mind though that _thefishbucket on the indexers will also need to be cleaned. It will retain that it has already seen the data, even if it was not indexed.
the command may not exists on the UF.
You can do the same by stopping splunk on the forwarder, deleting the folder $SPLUNK_HOME/var/lib/splunk/fishbucket, and restart splunk.
PS: every single log file will be re-indexed.
It will be the same command you used for cleaning the index, just use _thefishbucket after -index
splunk clean eventdata -index _thefishbucket
hey shane, i really dont understand.
can you update your answer with the command of implementing it.
hi i am following your post let me know what was the solution to your above mentioned question