Deployment Architecture

How to restrict big search on search head which consume most of CPU and memory.

msplunk33
Path Finder

Some users sending heavy, not fine tuned searches in search head cluster and this crash our search head server. How can restrict these kind of heavy searches which consume most of CPU and memory.

Labels (1)
Tags (1)
0 Karma

shivanshu1593
Builder

I faced a similar sort of problem with some users. Here's what I did in limits.conf. Please define it in $SPLUNK_HOME/etc/user/user_name/local/limits.conf.  Since this directory holds highest precedence for a user, their memory consumption will reduce. You'll have to do it for all the users that you want to restrict, manually.

If this seems too much of a manual work (though it's a highly effective and the best and recommended way), select an app, make it their default app and define these settings under /etc/apps/<their_default_app>/local/limits.conf. Please note, this will be effective for all the users, who will be using this app, including yourself. Also note, that if they use any other app to run and save their search, the setting would no longer be applicable to them (Hence applying to their user directories is the best solution)

enable_memory_tracker = true
search_process_memory_usage_threshold = <your desired value in MB. Defaults to 4000 (4 GB).>

 

For the part of CPU utilization, you can look into tuning their searches to run more efficiently, make them understand the value of effective searches. There's no limitation, as far as I know, that you can apply for CPU consumption. Though, reducing memory footprint will help with it a lot, as described above.

Let me know if it helps.

Thank you,

S

** If it helps. Please mark this as an accepted answer, as it helps future readers to get to the answers quickly **

Thank you,
Shiv
###If you found the answer helpful, kindly consider upvoting/accepting it as the answer as it helps other Splunkers find the solutions to similar issues###
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Check out the Workload Management feature at https://docs.splunk.com/Documentation/Splunk/8.0.6/Workloads/Aboutworkloadmanagement

---
If this reply helps you, Karma would be appreciated.
0 Karma

msplunk33
Path Finder

This documentation very good however it is so elaborate. Can I have a some easy mechanism to restrict the users memory and CPU usage with minimum config.

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...