Why does a scheduled saved search extract fewer results than the search that it is based on?

[andrewtrobec]

Hello,
I have a search and when I run it, it returns 514.299 events:

To speed up load times I have saved and scheduled that search, maintaining the same time range and extending the dispatch.max_count parameter in savedsearches.conf to 600.000 to ensure that no data is lost.

When I inspect the scheduled saved search execution, however, I notice that it doesn't return all of the results, even though it scans them:

This discrepancy can be as many as 30.000 results: it's never the same amount after every scheduled execution, and it never matches the results returned if I run the search independently..

Any ideas as to why this is happening? Any parameters I can check?

Thanks!

Andrew

[DalJeanis]

Okay, you need to figure out which records are dropped.  Here's how I would work this.

1) run each job for the exact same time range.

2) use |loadjob to load the output data from each job and cut it down to a few key fields.

3) use diff to compare the two files and find which records were added/deleted.

4) cut the file down to those records, look at just those records and see what's going on.

It may be that they are duplicate records that are being dropped somehow, or it may be that something in the search that you havent shown us has a reason to act differently