After patching activity, i have rebooted the Splunk cluster servers, ES search heads, Indexers and clustermaster, deployment server.
After rebooting, ES is not showcasing Notables. and it is not even showcasing results for notables macro.
Error coming as "Error in 'lookup' command: Lookups: The lookup table 'notable_xref_lookup' does not exist or is not available."
query: $SPLUNK_HOME/bin/splunk search "| rest /services/server/info splunk_server=* | fields splunk_server, kvStoreStatus"
I have checked the clustermaster for kvstore results:
deployment server failed
EnterPrise security Search Head Instances are-------- starting
idx 1,2,3,4,5,6,7 ready
notable macro is depend on notable_xref_lookup and notable_xref_lookup is KVStore lookup so you need to wait till your KVStore will be UP on ES Search Head Instances. If KVStore took too much time start then you need to check $SPLUNK_HOME/var/log/splunk/splunkd.log , $SPLUNK_HOME/var/log/splunk/mongod.log and $SPLUNK_HOME/var/log/introspection/kvstore.log & check why KVStore is taking more time to start.