Splunk Search

How do I resolve this message: "maximum number of concurrent auto-summarization searches on this instance has been reached"

frizzoS3
New Member

The below searches appear on my Skip Ration report with the following messages:
The maximum number of concurrent historical scheduled searches on this instance has been reached, and
The maximum number of concurrent auto-summarization searches on this instance has been reached.
I cannot locate these searches under the App to which they seem to belong to, nor am I finding them in Data Models.
Any suggestions on how to terminate these searches?
Thank you.
ACCELERATE_A3F1133B-692A-49B4-98B0-C6FC50DFB20D_splunk_app_stream_nobody_615f5f04b93533e7_ACCELERATE
ACCELERATE_DM_DA-ESS-ThreatIntelligence_Threat_Intelligence_ACCELERATE
ACCELERATE_DM_DellNetworking_Dell_Events_ACCELERATE
ACCELERATE_DM_SA-NetworkProtection_Domain_Analysis_ACCELERATE
ACCELERATE_DM_SA-ThreatIntelligence_Incident_Management_ACCELERATE
ACCELERATE_DM_SA-ThreatIntelligence_Risk_ACCELERATE
ACCELERATE_DM_SA-UEBA_UEBA_ACCELERATE
ACCELERATE_DM_cisco_ios_Cisco_IOS_Event_ACCELERATE

Tags (1)
0 Karma

harsmarvania57
Ultra Champion

Hi @frizzoS3,

Most of them are related to Datamodel acceleration.

First run below query to find SID of those searches:

index=_internal host=<search head> source=*scheduler.log* savedsearch_name=*ACCELERATE*

Once you able to find SID of those accelerated datamodels please run below query to find out exact splunk query and based on that you need to check it belongs to which datamodel.

index=_audit host=<search head> <SID>

And once you find it belongs to which datamodels then you can disable datamodel acceleration if it is not require. Once you disable datamodel accelerations all dashboards, searches which are using tstats summarised data to fetch data from datamodel acceleration summary will stop working, so be careful before disabling any datamodel accelerations.

Error which you are getting is due to you are running max searches on your splunk instance so you need to add more splunk search heads in your environment or you need to increase nuber if CPUs on existing search heads. Another option is find scheduled searches which are no more require and disable them.

I hope this helps.

Thanks,
Harshil

frizzoS3
New Member

Hi

I currently do not see any Data Model on the Indexer, or on the DS and SH.
Any other options to disabled these scheduled searches?

Regards

Frank

0 Karma

jmalherbe_splun
Splunk Employee
Splunk Employee

This is due to Data Model acceleration that's happening on the Indexers. I would suggest you review your accelerated Data Models to see if they're being used. Disable acceleration on those Data Models not being used and review the Constraints and Time Range accelerated for those that are being used.

If the Data Model Constraints are implemented via Macros or Event Types, it's useful to review those to see that they're properly constrained (for example limited to specific indexes), since you can't change the Constraints on Data Models themselves that are already accelerated without disabling the acceleration.

The reason why you want to review the constraints and time ranges, is to bring the time it takes to do the summaries down, so the accelerations don't run concurrently, so one can complete before the other starts.

0 Karma

frizzoS3
New Member

Hi

I currently do not see any Data Model on the Indexer, or on the DS and SH.
Any other options to disabled these scheduled searches?

Regards

Frank

0 Karma

jmalherbe_splun
Splunk Employee
Splunk Employee

The DM knowledge object lives on the the SH, and the Indexer serves the data which is why the acceleration happens there. Your searches being prefixed with "ACCELERATE_DM" indicates what it is. Go to "Settings --> Data models" on your SH to see a list, clicking on the first Input Field at the top allows you to filter according to the App context, and if you don't know which App the DM belongs to you can select "All" in this field.

In the "Data models" view, the DMs with a yellow lightning bolt next to them are the accelerated ones. Clicking on the arrow to the left of the DM name expands it to show details, and in there Summary Range indicates how long it's accelerated for. Clicking on "Edit --> Edit Acceleration" allows you to change the Summary Range, and clicking on either the name of the DM or on "Edit --> Edit Datasets" allows you to see the Constraints, usually implemented by Macros or Event Types.

If Constraints are present and implemented through Macros, you can go to "Settings --> Advanced search --> Search macros" to edit them, whereas if Constraints are implemented through Event types you can go to "Settings --> Event types" to edit them. You'd want to at least constrain them to specifying "index=", but more filtering may be needed depending on what's there already. Refer to Search/SPL best practice to guide you.

If you don't have access to any of these Settings views, you'll need to talk to your Administrator to get it sorted out.

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...