Why am I intermittently getting "no results found"

[CaptainHook]

Has anyone run up against an issue where you're intermittently getting a "no results found" on any search in their environment? If so, were you able to determine what was causing the issue?

As stated the issue is intermittent and to add to the complexity, I have noticed that one user may return results while another user cannot for the same search at the same time. (We are currently using SH pooling with 2 Searchheads). Nothing has been changed with the user roles prior to the issue surfacing and it has not been identified specifically to one or the other search heads.

Any thoughts and/or areas to focus on would be greatly appreciated. Thank you.

[CaptainHook · Accepted solution]

We are using search head pooling and it was determined that the authorize.conf file had a variation from one search head to the other, causing contention. Once the authorize.conf was updated to match and the search head restarted the issues resolved.

[sloshburch]

This could be a few things:

• Users are different roles. One role has access to the index the data lives in while the other doesn't.
• Role access definition is different between the two search heads - even if they are using same SHP, there could be system/local config that is not the same.

In both scenarios, using btool should show you the access. Run this on both search heads and look for differences in the properties srchIndexesAllowed, srchIndexesDefault, srchFilter for the role and the roles it importRoles from:

$SPLUNK_HOME/bin/splunk btool authorize list --debug

[CaptainHook]

Thanks Burch. I have compared the roles for our system/local as you have suggested and I am not seeing any major discrepancies within the roles. This is something that I had looked at in the GUI beforehand and we did have a slight difference within the srchIndexesAllowed. I have updated those to match (just to keep in uniform), but the issue seems to be more than a role issue.

[sloshburch]

Aight. Would you be able to show two examples (working and not working)? Keep the corporate stuff blurred but show the search string, time span, and job inspector heading that shows the # of results and events found as well as other errors and warnings.