We have a slightly odd architecture as we have a single search head, a single indexer and multiple forwarders (for the sake of arguement, all running 6.2.1 after several upgrades from prior versions). All of the current documentation only seems to address scenarios of a search head pool and indexer clusters, but nothing seems to allow for just one of each server type.
The issue I have is that the search head is indexing its own events (_audit, _internal and _introspection indexes are active), rather than forwarding them to the indexer. Is it just a simple matter of adding an outputs.conf to push all search head activity to the indexer? If so, what would I put in there?
Also, is there a way to make the indexer's indexes appear in the "Indexes" page of the search head, instead of seeing a pile of empty search head indexes?
Ok, so I guess this answers question one:
http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Forwardsearchheaddata
Not sure how I kept missing that. Still looking for help with question two.
We do something similar to what you are referring to.
Yes, outputs.conf on the searchhead to forward data to the indexer.
We have an indexapp that contains a local directory with indexes.conf in it. Whenever we add an index we update that file and deploy to all of our indexers and any searchhead who need the index list.
We use a script to "enable" the indexes without restarting the indexers.
echo Enter in Admin password
read pw
for indexer in indexer1 indexer2 indexer3
do
echo $indexer
echo -------------
curl -k -u admin:$pw https://$indexer:8089/servicesNS/admin/indexerbase/configs/conf-indexes/_reload
curl -k -u admin:$pw https://$indexer:8089/services/data/indexes/_reload
echo
sleep 1
done
echo -------------
echo Indexes reloaded
Ok, so I guess this answers question one:
http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Forwardsearchheaddata
Not sure how I kept missing that. Still looking for help with question two.
I guess I have basically figured question two out as well. Seems I can't find any answers until I've broadcast a question to the world. The index_status view (not sure how to access it without actually digging into the views via the User Interface settings screen) shows the pertinents of all available indexes.
../en-US/app/search/index_status