I want to get total memory allocated on 1 indexer and how much memory it is using. so that i could get remaining disk space left.
You can also use an mstats query to query to _metrics index:
| mstats latest(_value) as val WHERE index=_metrics AND metric_name=spl.intr.disk_objects.Partitions.data.* by data.mount_point, metric_name
| rename data.mount_point as mount_point
| eval metric_name=replace(metric_name,"spl.intr.disk_objects.Partitions.data.","")
| eval {metric_name}=val
| stats latest(*) as * by mount_point
| eval free = if(isnotnull(available), available, free)
| eval usage = round((capacity - free) / 1024, 2)
| eval capacity = round(capacity / 1024, 2)
| eval compare_usage = usage." / ".capacity
| eval pct_usage = round(usage / capacity * 100, 2)
| stats first(compare_usage) AS compare_usage first(pct_usage) as pct_usage by mount_point
| rename mount_point as "Mount Point", compare_usage as "Disk Usage (GB)", pct_usage as "Disk Usage (%)"
Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards
Will
Hi @SN1
Some good answers here, its worth noting that for me
| rest /services/server/status/partitions-spacedoesnt give me the right data, and it can depend on how your partitions are configured (e.g. multiple partitions for hot/warm/cold etc)
If you're using Linux then its worth also checking something as simple as in the linux command line
df -hThis will list all the filesystems on the server and show you the size, used and available disk space.
I'd definitely recommend setting up some proper monitoring using the Splunk TA for *Nix to cover your servers and cover all partitions and filesystems.
Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards
Will
The difference between disk usage and memory has already been pointed out.
There is also one more thing worth noting - the disk utilization on indexers is usually managed by adjusting retention parameters (you might also get some additional usage from knowledge bundles and intermediate results but they are rarely very significant). And the memory usage can vary greatly depending on the current load at the time of checking since memory is used mostly for searching. So the more more complicated searches you're running at any given moment, the higher memory usage.
| rest /services/server/status/partitions-space splunk_server=*
| eval totalGB = round(capacity/1024/1024, 2)
| eval freeGB = round(free/1024/1024, 2)
| eval usedGB = round((capacity - free)/1024/1024, 2)
| table splunk_server, totalGB, usedGB, freeGB
To get the total memory allocated on an indexer and its current usage (which is different from disk space), you can use the following Splunk commands:
For memory information:
This will show you key metrics including:
If you're specifically interested in Splunk's memory usage:
For disk space information (which seems to be what you're actually asking about):
For specific index volume usage:
Note that memory usage and disk space are different resources. Memory refers to RAM available for processing, while disk space refers to storage capacity for data. Your question mentions memory but ends with disk space, so I've provided commands for both.
when i am running this search it is giving 16GB as total_GB while our total size is 16Tb.
Hi @SN1
This is because the values from the endpoint are in MB but are being divided by 1024 twice in this search hence they become in TB.
try switching 1024/1024 for just 1024 in each occurrence and see if that resolves for you 🙂
Will
Hi @SN1 ,
in addition to the perfect answer of @kiran_panchavat ,
you could install the Splunk_TA_nix add-on ( https://splunkbase.splunk.com/app/833 ) and extract additional information from the linux system you're using.
Ciao.
Giuseppe
Well... TA_nix without careful tweaking what it reports can be a handful. It's just a bunch of ziptie and duct-tape connected scripts giving you some relatively unfriendly output. And if you just install it and enable all inputs, it can get noisy.