We have had several examples recently where scheduled searches appear to run in the _internal log, complete successfully, and find no results when results were available. Using _indextime vs _time to determine the lag on the results it seems like there is no reason why the results were not found by the search. We have an open case with support but this makes me wonder: Where does _indextime fall in the indexing process? Is it when the event is written to the first bucket? Is it when the first searchable copy is written? Is it when the full replication/search factors have been met across the cluster (unlikely)? As you can probably guess I'm wondering if we can get into a state where data is written/indexed but not actually searchable. For reference, we have a multisite peer cluster with 8 members over 2 sites (2 AZs in the same AWS region). Thanks! ... View more

We moved from a single search head set up to a Search Head cluster running 6.2.1. We are manually doing cleanup on knowledge objects created pre-cluster that can't be deleted or have permissions changed via the UI (a known, annoying bug). We have had pretty good success deleting these searches from the appropriate /etc/shcluster savedsearches.conf on the deployer and doing a bundle push. I tried to do the same thing with dashboards today, deleting the XML files on the deployer. This works with dashboards that were under the app context (i.e. /etc/shcluster/apps/search/local/data/ui/views/ ). This does not work with dashboards that were under the user context (i.e. etc/shcluster/users/USERNAME/search/local/data/ui ). The file is deleted from the deployer, but remains in the default dir on the search heads. The docs are not particularly clear on this, it does seem like the deployer logic for pushing out /shcluster/users is different than shcluster/apps. I'm assuming only .conf files under /shcluster/users get updated and deletions don't propagate? Have others tried this? Any tricks? Am I stuck manually deleting the files on every search head? ... View more