I noticed today that there seems to be a lot more pids running by Splunk in 6.5. Is this by design? Below is an example - I don't recall seeing these "launcher" pids before.
splunk 1971 1 99 21:23 ? 00:10:19 splunkd -p 8089 restart
splunk 1978 1971 0 21:23 ? 00:00:00 [splunkd pid=1971] splunkd -p 8089 restart [process-runner]
splunk 5953 1978 0 21:30 ? 00:00:01 [splunkd pid=1971] [search-launcher]
splunk 5954 5953 0 21:30 ? 00:00:00 [splunkd pid=1971] [search-launcher] [process-runner]
splunk 5956 1978 0 21:30 ? 00:00:01 [splunkd pid=1971] [search-launcher]
splunk 5964 5956 0 21:30 ? 00:00:00 [splunkd pid=1971] [search-launcher] [process-runner]
You seeing this on SH, Indexer, Utility, everything?
Honestly that seems fishy that they are the same. Did they clear after restart or are they persisting?
Compared to my DS, I only see relevant looking pids.
I'd suggest stopping splunk and seeing is there's any remaining pids. If so, kill those zombies (head shot FTW) and then start splunk and I'm guessing you're good. Sometimes unix processes just live on so I've seen that quirk before.
This looks OK to me. The
[splunkd pid=1971] shows that those processes tie back to the main splunkd launch process. The PID chain looks accurate.
5954 and 5964 may be the two default DMA (data model acceleration) search processes that were introduced in 6.5, but I am not 100% certain.