I have a linux host running 6.0.2 and I notice that the number of splunkd rising on one of my indexers. When the processes get above 100+ splunkd processes everything starts to suffer. Has anyone seen this? Under what circumstances does splunk start to spawn more processes?
normally there are three kind of splunkd processes around which can be shown by using the $SPLUNK_HOME/bin/splunk status command. You will get a list of PID's for the main Splunk process, Splunk Web process and Splunk helper processes.
Those helper processes are mainly your searches, so if you see the amount of helper processes raising means you run more searches. Form the command output you can use the PID and check what searches are running or you use the S.o.S app and check your searches from there.