Why does a scheduled saved search extract fewer results than the search that it is based on?


I have a search and when I run it, it returns 514.299 events:

alt text

To speed up load times I have saved and scheduled that search, maintaining the same time range and extending the dispatch.max_count parameter in savedsearches.conf to 600.000 to ensure that no data is lost.

When I inspect the scheduled saved search execution, however, I notice that it doesn't return all of the results, even though it scans them:

alt text

This discrepancy can be as many as 30.000 results: it's never the same amount after every scheduled execution, and it never matches the results returned if I run the search independently..

Any ideas as to why this is happening? Any parameters I can check?



0 Karma


Okay, you need to figure out which records are dropped.  Here's how I would work this.

1) run each job for the exact same time range.

2) use |loadjob to load the output data from each job and cut it down to a few key fields.

3) use diff to compare the two files and find which records were added/deleted.

4) cut the file down to those records, look at just those records and see what's going on.


It may be that they are duplicate records that are being dropped somehow, or it may be that something in the search that you havent shown us has a reason to act differently 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...