Splunk Enterprise

Why does my scheduled saved search randomly decide to return no results?

andrewtrobec
Motivator

Hello!

I noticed that one of my scheduled saved searches randomly refuses to return results.  I can run the search at any point from the search bar and get data, even immediately after the scheduled saved search returns 0.  Here are the results of when it was scheduled at 2 and 5 minute intervals:

andrewtrobec_1-1593504439156.png

Randomly it will conclude with 0 results after a second with no errors.

Why would it do this?  How can I ensure that the results are produced consistently each time?

Thanks!

Andrew

Labels (1)
0 Karma

anilchaithu
Builder

@andrewtrobec 

Did you check search.log for any errors?

0 Karma

andrewtrobec
Motivator

@anilchaithu 

Thanks for the reply.  The search.log logs do not contain any errors, but when I compare them for an execution that contains results (successful) vs. an execution that does not contain results (failed), I see the following differences:

  • successful execution sets user context before the following line, the failed execution sets it afterwards:

07-01-2020 06:00:07.397 INFO SearchParser - PARSING: | streamnoop

  • successful execution has the following line more than 150 times, the failed execution has it twice:

07-01-2020 05:50:04.890 INFO SearchEvaluator - using old evaluator

  • successful execution has the following line as the final line in the log, the failed execution does not:

07-01-2020 05:51:43.061 INFO PipelineComponent - Process delayed by 97.988 seconds, perhaps system was suspended?

Other than that they're pretty much the same.

Regards,

Andrew

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...