Monitoring Splunk

Monitoring Console shows inconsistent max memory on Indexer

afx
Contributor

Hi,
on a 7.2.4 Cluster my Indexers show memory usage of more than 80% in the initial screen of the monitoring console.
When I then go into the Resource Usage: Machine screen, I see that Splunk knows it has 7GB memory. in the Machine information on top, which also corresponds to the free command on the box.
But when I look at the snapshot or memory usage graph, I see nonsense:
I see 35.000 of 40.000 MB used.
Where does this come from and how can it be fixed?

thx
afx

0 Karma

harsmarvania57
Ultra Champion

I am looking at 7.2.7 Monitoring Console and in Resource Usage: Machine -> Snapshot -> Memory Usage (MB) is running REST API search | rest splunk_server=indexer_host /services/server/status/resource-usage/hostwide . Can you please try to run this search and check what it returns.

0 Karma

afx
Contributor

This returns nonsense for mem as well:
mem: 40892.957
men_used: 36003.289

thx
afx

0 Karma

harsmarvania57
Ultra Champion

Have you tried to restart splunk on Indexer ? If yes and it will not solve problem then I'll suggest to open case with splunk.

0 Karma

afx
Contributor

A rolling restart fixed this, thx!
But now I get
Some Data is Not Searchable
Search Factor is Not Met
Replication Factor is Not Met
And it seems that this comes from _audit.
Had that last year, it seemd to have resolved itself.

cheers
afx

0 Karma

harsmarvania57
Ultra Champion

Yes it will resolve automatically, that is due to bucket fixup activites.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...