How to get ID of splunkd process executing saved search?

[YuriSpirin]

Hello!

I'm trying to resolve issues with splunkd being killed by OOM Reaper and it would be nice to know which saved search (or ad-hoc search) is consuming too much RAM. In Linux messages from Search Head I have a PIDs of reaped splunkd processes and the question is how to get PID of splunkd for particular saved search from _internal index. Scheduler events have SID field like this:

sid="scheduler_aS5zLnNva29sb3Y_czdfc2llbV9uZXR3b3Jr__RMD58313482a27867d57_at_1716903900_27923"

Is the last part of SID (27923) a Linux process ID? Or maybe I can get PID from some other source?

[yuanliu · Accepted solution]

If you already know the search ID, it shouldn't matter whether it is saved search or not.

| rest /services/search/jobs/<search ID>
| fields sid pid

Here, pid is the process ID.  There are tons of other output from this call.

If you only know the saved search's name, you can search by label

| rest /services/search/jobs
| where isSavedSearch == 1 AND label = "<your report name>"
| fields label sid pid

Hope this helps.

[YuriSpirin]

Thanks, that was helpful!