I have AIX servers on which Splunk universal forwarders are installed where splunkd process suddenly consumed high memory around 95%. After investigation, it's been found that there isn't much increase in number of files processed. I have tried using *nix app to get the resource usage of AIX servers but that has made the situation even worse. Since, there are multiple commands that get fired in the background, the AIX system crashed. What's the best way to get the resource usage of AIX servers?
You can use the nmon application and the related TA-nmon on Linux/AIX/Solaris for monitoring if you are not happy with the TA-nix app.
Note that it will monitor a lot more than just Splunk as it's a generic OS monitoring, and it has a lot of depth in terms of the monitoring data that it will create...
I've used this on both AIX & Linux
What are the exact commands that you tried to run on the *nix app ? There are multiple commands which are specific to the linux. If the same has been enabled to run on the AIX servers then it's likely that the cpu and mem shoots up.
Thank you for your prompt response. I surely agree that there were multiple linux specific commands running behind that had caused the issue and I'm looking further for a solution on how to monitor the resource usage in Splunk.
Would certainly appreciate your help!