Knowledge Management

Why are only 10,000 events making it into the summary index?

Dan
Splunk Employee
Splunk Employee

I'm having an issue with my summary index. I have a search which results in 48000+ events. I saved the search and enabled summary indexing on it. My issue is that only 10,000 events get put into the summary index each time the search runs.

Any idea what limit I need to change to get all my results in my summary index? The saved search is similar to this:

index="myidx" | rex field=source "data/(?<program_name>\w+)/(?<receive_date>\w+)___" | dedup source | stats list(program_name) as programName, list(receive_date) as receiveDate by source
1 Solution

Dan
Splunk Employee
Splunk Employee

There's by default a limit of 10,000 events that will get summary indexed from each run of a scheduled search. This default limit will actually be removed in most cases in the next maintenance release (4.1.2). For now, here's a couple of workarounds.

1) in savedsearches.conf for the app that your search belongs to, under the stanza for your scheduled searches, add dispatch.max_count=100000 (or whatever limit you want) ALSO, in etc/system/local/limits.conf (create it if it doesn't exist), under the [scheduler] stanza, set max_action_results=100000 (or a limit of your choosing).

OR

2) instead of setting enabling the summary indexing action of the saved searches, explicitly add a " | collect" to the end of your saved searches. This will change the search itself to directly populate your summary index. (instead of the default behavior, which is that the scheduler reads the result of your search and then populates the summary index from that result)

View solution in original post

Dan
Splunk Employee
Splunk Employee

There's by default a limit of 10,000 events that will get summary indexed from each run of a scheduled search. This default limit will actually be removed in most cases in the next maintenance release (4.1.2). For now, here's a couple of workarounds.

1) in savedsearches.conf for the app that your search belongs to, under the stanza for your scheduled searches, add dispatch.max_count=100000 (or whatever limit you want) ALSO, in etc/system/local/limits.conf (create it if it doesn't exist), under the [scheduler] stanza, set max_action_results=100000 (or a limit of your choosing).

OR

2) instead of setting enabling the summary indexing action of the saved searches, explicitly add a " | collect" to the end of your saved searches. This will change the search itself to directly populate your summary index. (instead of the default behavior, which is that the scheduler reads the result of your search and then populates the summary index from that result)

View solution in original post

Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!