topic Re: Monitoring Console shows inconsistent max memory on Indexer in Monitoring Splunk

Monitoring Console shows inconsistent max memory on Indexer

afx — Fri, 05 Jun 2020 23:23:31 GMT

Hi,
on a 7.2.4 Cluster my Indexers show memory usage of more than 80% in the initial screen of the monitoring console.
When I then go into the Resource Usage: Machine screen, I see that Splunk knows it has 7GB memory. in the Machine information on top, which also corresponds to the free command on the box.
But when I look at the snapshot or memory usage graph, I see nonsense:
I see 35.000 of 40.000 MB used.
Where does this come from and how can it be fixed?

thx
afx

Re: Monitoring Console shows inconsistent max memory on Indexer

harsmarvania57 — Wed, 01 Apr 2020 09:26:34 GMT

I am looking at 7.2.7 Monitoring Console and in Resource Usage: Machine -> Snapshot -> Memory Usage (MB) is running REST API search | rest splunk_server=indexer_host /services/server/status/resource-usage/hostwide . Can you please try to run this search and check what it returns.

Re: Monitoring Console shows inconsistent max memory on Indexer

afx — Wed, 01 Apr 2020 09:46:08 GMT

This returns nonsense for mem as well:
mem: 40892.957
men_used: 36003.289

thx
afx

Re: Monitoring Console shows inconsistent max memory on Indexer

harsmarvania57 — Wed, 01 Apr 2020 09:49:48 GMT

Have you tried to restart splunk on Indexer ? If yes and it will not solve problem then I'll suggest to open case with splunk.

Re: Monitoring Console shows inconsistent max memory on Indexer

afx — Wed, 01 Apr 2020 10:36:07 GMT

A rolling restart fixed this, thx!
But now I get
Some Data is Not Searchable
Search Factor is Not Met
Replication Factor is Not Met
And it seems that this comes from _audit.
Had that last year, it seemd to have resolved itself.

cheers
afx

Re: Monitoring Console shows inconsistent max memory on Indexer

harsmarvania57 — Wed, 01 Apr 2020 10:49:03 GMT

Yes it will resolve automatically, that is due to bucket fixup activites.