I have dashboards that have scheduled saved searches. The search permissions are set to app with appropriate user permissions. When I load the dashboard I can see that the search ran 9 hours ago but shows no results. If I click on view results there are truly no results BUT if I rerun the search I get immediate results.
The thing is that this is spordic. Sometimes I have no trouble and sometimes none of the searches on a dashboard will show results. But every time I rerun them I get results.
Run the search as a power user who has access to all indexes & see what index the data is coming from. Then check what roles the user who owns the saved search has to see if the correct index is in "Indexes searched by default."
I had several saved searches that were created by a user who had default search on index "blahblahblah." When he left the company, the saved searches were transferred to another user so they would keep running. Unfortunately, the transferred-to user did not have "blahblahblah" as a default search index so the saved search always came up blank. If I clicked on the "View Results in Splunk" link in the email it showed nothing, however if I then hit the Search button from those results with my elevated access it searched all indexes including "blahblahblah." Adding "blahblahblah" to the transferred-to user's default indexes corrected the problem.
Ayn, could a too-short ttl be the cause if he is able to retrieve those results through "view results" at the same time his dashboard doesn't? By that I mean: why would view results ignore a too-short ttl?
I have the same problem here.
I got several scheduled searches which show no results on the dashboards, if I click the view results link and just hit the search bnutton - without changing anything - my results appear as expected.
I run the following search:
`index=_internal source=*scheduler.log | eval sched = strftime(scheduled_time, "%Y-%m-%d %H:%M:%S") | table sched status savedsearch_name`
Which shows me success status for all the scheduled searches. Strange thing about this is the searches all worked before. They're also running on a dedicated Job-Server which is just running these searches.
Any ideas how to get the problem identified?