Do the following steps:
1) Disable the applications on the servers with Forwarders using the Deployment manager (or manually do so using the GUI)
2) On the Index server, stop Splunk and clean all event data for a given index:
/opt/splunk/bin/splunk stop
/opt/splunk/bin/splunk clean eventdata dev_tools
3) On the servers with Splunk Forwarders, stop and clean all:
/opt/splunk/bin/splunk stop
/opt/splunk/bin/splunk clean all
4) Start the Splunk Indexer
/opt/splunk/bin/splunk start
5) Verify the index you just cleared is empty and is staying empty before proceeding. If it isn't staying empty ensure you've disabled the applications on the servers with Splunk Forwarders and make sure you didn't overlook any servers.
6) Start the Splunk forwarder servers:
/opt/splunk/bin/splunk start
6) Re-enable the applications via the Deployment Manager or manually using the Agents GUI.
Note: Be aware, if you "clean all" on the forwarders you will also clear out the pointers for all indexes and applications so this process is only good to completely re-index an environment environment or set of indexes a set of servers and indexer may contain.
Note 2: Don't do this unless your license (and hardware) can handle the spike as you will most certainly index a lot of legacy data if your inputs are setup with wildcard inputs.
... View more
