Splunk Enterprise Security

High CPU and memory usage with ESS (Enterprise Security Suite)

LukeMurphey
Champion

I am experiencing high CPU and memory usage with ESS. In some case, the resource usage is high enough to cause Splunk to crash. How can I fix this issue?

1 Solution

LukeMurphey
Champion

Synopsis:

Adding the notable index to the "Indexes searched by default" may cause correlation searches to enter a feedback loop that causes excessive resource usage.

Description:

This issue can be caused when the the notable index is indexes to be searched by default. In particular, this causes some of the ESS's correlation searches to trigger on their own findings since correlation searches included information about what originally triggered them in the notable index. Adding the notable index to the default searches indexes causes correlation searches to re-detect another finding based on the content of a prevous correlation search firing.

Solution:

The solution is to remove the notable index from the list of indexes to be searched by default.

Fixing via the CLI:

To fix the issue via the CLI, open or create the file $SPLUNK_HOME/etc/system/local/authorize.conf and change the "srchIndexesDefault" parameter for each role to exclude the notable index. Below is a sample config:

[role_admin]
srchIndexesDefault = main

Fixing via the GUI:

To fix the misconfiguration through the SplunkWeb interface use the steps defined below:

  1. Open the Manager (through the link in the top right of SplunkWeb)
  2. Open the configuration page for user roles by navigating to: Manager » Access controls » Roles
  3. For each role: remove "notable" from the "Indexes searched by default" option

View solution in original post

LukeMurphey
Champion

Synopsis:

Adding the notable index to the "Indexes searched by default" may cause correlation searches to enter a feedback loop that causes excessive resource usage.

Description:

This issue can be caused when the the notable index is indexes to be searched by default. In particular, this causes some of the ESS's correlation searches to trigger on their own findings since correlation searches included information about what originally triggered them in the notable index. Adding the notable index to the default searches indexes causes correlation searches to re-detect another finding based on the content of a prevous correlation search firing.

Solution:

The solution is to remove the notable index from the list of indexes to be searched by default.

Fixing via the CLI:

To fix the issue via the CLI, open or create the file $SPLUNK_HOME/etc/system/local/authorize.conf and change the "srchIndexesDefault" parameter for each role to exclude the notable index. Below is a sample config:

[role_admin]
srchIndexesDefault = main

Fixing via the GUI:

To fix the misconfiguration through the SplunkWeb interface use the steps defined below:

  1. Open the Manager (through the link in the top right of SplunkWeb)
  2. Open the configuration page for user roles by navigating to: Manager » Access controls » Roles
  3. For each role: remove "notable" from the "Indexes searched by default" option

rroberts
Splunk Employee
Splunk Employee

Solved it for me!

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...