Splunk support solved. Tenable app had loaded a conf that sent notable events to the stash bypassing the notable index. Disabled bad conf lines and it is back up ... View more

I put the last comment i the answer posts, here is the savedsearches.conf: [Event Sequencing Engine - Main] action.keyindicator.invert = 0 action.makestreams.param.verbose = 0 action.nbtstat.param.verbose = 0 action.notable.param.verbose = 0 action.nslookup.param.verbose = 0 action.ping.param.verbose = 0 action.risk.param.verbose = 0 action.threat_add.param.verbose = 0 alert.track = 0 enableSched = 1 ... View more

After eliminating search related instances this is the only event occuring: 08-05-2019 07:42:01.526 -0400 ERROR DatabaseDirectoryManager - Failed to remove summary of bid=notable~2~324931C1-6BA3-4CAF-A0E1-B88B4457E0BE with cid="dma|notable~2~324931C1-6BA3-4CAF-A0E1-B88B4457E0BE|FE9350B2-8BA7-4622-886B-62F14F94F944_DM_Splunk_SA_CIM_Authentication" from summary manager, skipping remove. ... View more

My last comment did not post: No event in the summary index - all time index=_* (WARN OR ERROR) notable* #-- results: No warn, error=0 on all events, hit=0, miss=0 Values Count % #--removed count and % in results splunkd_remote_searches audittrail splunkd contentinfo_rest_handler scheduler splunkd_access splunk_disk_objects configuration_check splunk_resource_usage [Event Sequencing Engine - Main] action.keyindicator.invert = 0 action.makestreams.param.verbose = 0 action.nbtstat.param.verbose = 0 action.notable.param.verbose = 0 action.nslookup.param.verbose = 0 action.ping.param.verbose = 0 action.risk.param.verbose = 0 action.threat_add.param.verbose = 0 alert.track = 0 enableSched = 1 ... View more

Many entries, no WARN, error=0, hit=0, miss=0 in each result savedsearches.conf [Event Sequencing Engine - Main] action.keyindicator.invert = 0 action.makestreams.param.verbose = 0 action.nbtstat.param.verbose = 0 action.notable.param.verbose = 0 action.nslookup.param.verbose = 0 action.ping.param.verbose = 0 action.risk.param.verbose = 0 action.threat_add.param.verbose = 0 alert.track = 0 enableSched = 1 ... View more

I have tested 4 manual correlation searches and do get results from the ES Search bar I ran the Untriaged Notable Events Correlation Search but got no results, I am assuming because the notables are not showing up. Up to 10 successful manual Correlation searches which are enabled. ... View more

Disabled all correlation searches, none are real time, validated received events. It took a week to get up to 100 correlations by testing about 10 a day, some returned little some a lot. I have now enabled 26 correlations for about 2 hours and still no notables in Security Posture. I have also validated all indexes are owned by the splunk process running the service. ... View more

My ES shows a skip ratio of 0.80% Notable index is healthy I am using an account with Splunk Admin and ess_admin permissions The server and indexes pass health checks. The Security Posture dashboard displays "0"s but I do see notables in some of the Security Domains options. ... View more