Splunk Dev

Saved Search not returning all events

varun85negi
Engager

Hi All,

We have a Splunk saved search in which we are querying same index but multiple sourcetypes like below:

(Index=A sourcetype=A1) OR (index=A (sourcetype=A2 OR sourcetype=A3 OR sourcetype=A4))

Issue is sometimes the events from 2nd part is not returned (Sourcetype A2, A3 and A4) but events from 1st part is correctly returned.

Below are some messages from internal logs:

search_type=" scheduled ", user="X", app="X", savedsearch_name="X", priority=default, status=continued, reason="The maximum number of concurrent running jobs for this historical scheduled search on this cluster has been reached", concurrency_category="historical_scheduled", concurrency_context="saved-search_cluster-wide", concurrency_limit=1, scheduled_time=1521180300, window_time=0

search_type="scheduled", user="X", app="X", savedsearch_name="X", priority=default, status=continued, reason="The maximum number of concurrent historical scheduled searches on this cluster has been reached", concurrency_category="historical_scheduled", concurrency_context="cluster-wide", concurrency_limit=69, scheduled_time=1521180300, window_time=0

search_type="scheduled", user="X", app="X", savedsearch_name="X", priority=default, status=continued, reason="The maximum number of concurrent historical scheduled searches on this cluster has been reached", concurrency_category="historical_scheduled", concurrency_context="cluster-wide", concurrency_limit=69, scheduled_time=1521180300, window_time=0

Tags (1)
0 Karma

Azeemering
Builder

How often do you run this saved search?
If a saved search is scheduled to run every minute, but it takes five minutes to run, then it will skip the next four times that it tries to run....

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

A Four-Part Event Series: Full Stack Observability For the AI Era

As AI reshapes applications, infrastructure, and the way teams operate, the traditional boundaries of ...

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Event Series: Level up your SOC: Advancing with Splunk Enterprise Security

AI has fundamentally raised the stakes for security operations, and this three-part series is your guide to ...