Monitoring Splunk

Why UF consuming so much swap

tkw03
Communicator

Hello

Trying to figure out why my UF is consuming 37GB of swap space

Ran some commands and here are the results

[splunk@server07 ~]$ free -h
             total       used       free     shared    buffers     cached
Mem:           94G        93G       1.3G        46G       252M        49G
-/+ buffers/cache:        43G        51G
Swap:          57G        53G       4.2G

The swap calcultaions by splunk process:

[splunk@server7 ~]$ grep --color VmSwap /proc/100427/status
VmSwap:     4180 kB
[splunk@server7 ~]$ grep --color VmSwap /proc/100423/status
VmSwap: 37438788 kB

Anyone have any ideas why its consuming so much swap?
This doesn't seem normal

Thanks for the thoughts!

0 Karma
1 Solution

tkw03
Communicator

Went ahead and uploaded a case to support. Will close this. Will update once support gives an answer.
Thanks everyone

View solution in original post

0 Karma

tkw03
Communicator

Went ahead and uploaded a case to support. Will close this. Will update once support gives an answer.
Thanks everyone

0 Karma

tkw03
Communicator

Looks like it was some sort of cache issue. Rolled the Splunk service and it released the swap space.
Thanks!

0 Karma

pruthvikrishnap
Contributor

This is too high, i would recommend you to raise a support request to check why is it consuming 37GB of space.
this link give you some understanding https://docs.splunk.com/Documentation/Splunk/7.2.4/Troubleshooting/Troubleshootmemoryusage

ddrillic
Ultra Champion

One thread about such cases - Why does an AIX 6.5.2 forwarder have high swap/memory and cpu consumption?

All experts say -

-- High numbers of monitored files can cause such behavior .

0 Karma

tkw03
Communicator

Is there a way, other than using the localhost:8089 endpoint as descibed in the link you attached, to find the number of monitored files? The 8089 endpoint on the UF is disabled.
Like I said CPU isnt spiking

0 Karma

BainM
Communicator

What is it doing? How many apps are installed on it? What does the log tail show? Any errors? Is it chewing up CPU?

0 Karma

tkw03
Communicator

The biggest thing there is the Nix app. other than that not many apps. Splunkd is using on average 30% of a single core. So not something I see as an issue. Tailing the splunkd log only shows connections to indexers. NO errors.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...