What are the best practices for configuring Splunk memory and swap partition space? 

[jiaminyun]

Hello, What are the best practices for configuring Splunk memory and swap partition space? now resources is: The resources of the three index nodes are 24C, 64GB, 2T, and SSD with a 10 gigabit transmission rate. Each index node has 64GB of physical memory, and SWAP has 8GB. SWAP strategy requires physical memory to exceed 70% before it can be used. The current situation is that only 1.6GB of physical memory is used, but the swap uses 3.8GB. The following is the alarm information. [Alarm Name] system.swap.used_pct [Warning content] The usage rate of swap partition has reached 39.76%,. and the AVG has exceeded the threshold of 20.0% in the past minute. I have some questions to ask: 1. Why is swap usage so much higher than memory. 2. How to configure memory and swap partition space, and what are the best practices?

[PickleRick]

Apart from very specific cases of systems which have constant memory requirements, there is no way of telling how much resources you will need especially not knowing your load, your data and so on.

Having said that - there are general sizing hints there https://docs.splunk.com/Documentation/Splunk/latest/Capacity/Referencehardware

Additionally, IMHO indexers should not use swap. At all. If you have to reach to disk for "memory", that means you're slowing your I/O which means you're building up your queues which means you're using up even more memory. That's a downhill path. (ok, you can have a very small swap to keep some sleeping system daemons out of active RAM but that's usually not worth the bother).

[dural_yyz]

You haven't mentioned anything about which OS specifically and what else is or may be using resources.  Since your system exceeds minimum recommendations I would look for the total package.  You may need an OS expert and not a Splunk expert to help track this down.

[jiaminyun]

We are trying to modify vm.swappiness to 10 in/etc/sysctl.conf and are still observing. What is the appropriate size for allocating swaps to a single node in Splunk? We observe that most of them are used more frequently by mongols

[jiaminyun]

Thank you for your reply,
1. Splunk uses the Galaxy Kirin V10 SP2 X86 version for the three index nodes
2. The current resource utilization situation is:
Swap parameter vm. swap=30
3. The status is that only 1.6GB of 64GB memory was used, but SWAP used nearly 4GB