Monitoring Splunk

Monitoring Console shows inconsistent max memory on Indexer

afx
Contributor

Hi,
on a 7.2.4 Cluster my Indexers show memory usage of more than 80% in the initial screen of the monitoring console.
When I then go into the Resource Usage: Machine screen, I see that Splunk knows it has 7GB memory. in the Machine information on top, which also corresponds to the free command on the box.
But when I look at the snapshot or memory usage graph, I see nonsense:
I see 35.000 of 40.000 MB used.
Where does this come from and how can it be fixed?

thx
afx

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

I am looking at 7.2.7 Monitoring Console and in Resource Usage: Machine -> Snapshot -> Memory Usage (MB) is running REST API search | rest splunk_server=indexer_host /services/server/status/resource-usage/hostwide . Can you please try to run this search and check what it returns.

0 Karma

afx
Contributor

This returns nonsense for mem as well:
mem: 40892.957
men_used: 36003.289

thx
afx

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Have you tried to restart splunk on Indexer ? If yes and it will not solve problem then I'll suggest to open case with splunk.

0 Karma

afx
Contributor

A rolling restart fixed this, thx!
But now I get
Some Data is Not Searchable
Search Factor is Not Met
Replication Factor is Not Met
And it seems that this comes from _audit.
Had that last year, it seemd to have resolved itself.

cheers
afx

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Yes it will resolve automatically, that is due to bucket fixup activites.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...