topic Re: Perfmon:Memory missing from search head in Getting Data In

Perfmon:Memory missing from search head

dtrelford — Sat, 16 Jun 2018 22:02:38 GMT

I'm trying to timechart memory usage on my search head, but for some reason it's not collecting data. Specifically, memory counters. Other perfmon counters (such as cpu and disk) are available. Checked inputs.conf and verified settings match other servers that are correctly sending memory counter data to the indexers:

[perfmon://Memory]
counters = Page Faults/sec; Available Bytes; Pages/sec; Page Reads/sec; Page Writes/sec
disabled = 0
interval = 10
object = Memory
useEnglishOnly=true
index = perfmon

I have run lodctr and tested the counter in Performance Monitor on the server itself, so I know the counter works locally. Where else can I look for a fault?

EDIT:
To clarify- all other counters configured in inputs.conf are working.
perfmon://Memory is working on other servers it's deployed on. Search Head is the only server where the memory counter is missing from search results.

Re: Perfmon:Memory missing from search head

soumyasaha25 — Tue, 19 Jun 2018 10:05:19 GMT

can you check if you have the input OperatingSystem enabled in your inputs.conf
sample stanza below
[WinHostMon://OperatingSystem]
interval = 600
disabled = 1
type = OperatingSystem
index = windows

i had a similar issue long time back enabling this input resolved it. Hope it works for you as well

Re: Perfmon:Memory missing from search head

dtrelford — Tue, 19 Jun 2018 13:38:41 GMT

Hi soumyasaha,

The stanza you provided has disabled = 1, is that correct? Wouldn't that disable that input?

Re: Perfmon:Memory missing from search head

soumyasaha25 — Tue, 19 Jun 2018 14:40:36 GMT

yes, sorry. we had disabled it later. can you try to put the stanza with disabled = 0

Re: Perfmon:Memory missing from search head

dtrelford — Tue, 19 Jun 2018 18:22:24 GMT

Added the stanza and restarted splunk on the search head. Unfortunately it did not resolve the problem. Search head memory counters are still missing.

Re: Perfmon:Memory missing from search head

dtrelford — Tue, 29 Sep 2020 20:58:59 GMT

After working with Splunk Support, I found the solution.

There is a default app created by splunk - Splunk_TA_microsoft_ad

Although all the inputs used in this app were disabled, the app itself was enabled in the app.conf. This was overriding our setup in a custom perfmon app.

Changes made to resolve:

.../etc/apps/Splunk_TA_microsoft_ad/local/app.conf
[install]
state = disabled

Restarted Splunk. Issue is now resolved.