Splunk ES Notable events not getting triggered

[diwakar]

Hello Everyone, I'm trying to use Splunk ES feature for AWS cloudtrail data. I'm using default main index for cloudtrail events. I created one correlation search and added trigger action as notable on SH. I see Alerts getting fired but no notable events created but the below error message is displayed at each trigger.

Search peer ip-10-5-2-15.ec2.internal has the following message: Received event for unconfigured/disabled/deleted index=cim_modactions with source="source::/opt/splunk/var/log/splunk/notable_modalert.log" host="host::splunksearch" sourcetype="sourcetype::modular_alerts:notable". So far received events from 3 missing index(es). 

Search peer ip-10-5-2-204.ec2.internal has the following message: Received event for unconfigured/disabled/deleted index=notable with source="source::Threat - AWS CreateAccessKey Test - Rule" host="host::splunksearch" sourcetype="sourcetype::stash". So far received events from 2 missing index(es). 

I see notable index being shown [Indexes and Volumes: Instances] same as main index but without any event. Could anyone help me resolve this problem.

[diwakar · Accepted solution]

Creating Notable index on cluster master solved the problem.

[notable]
homePath = $SPLUNK_DB/notabledb/db
coldPath = $SPLUNK_DB/notabledb/colddb
thawedPath = $SPLUNK_DB/notabledb/thaweddb

I followed this post : https://community.splunk.com/t5/All-Apps-and-Add-ons/How-to-create-a-new-index-in-index-cluster-6-2-...

[richgalloway]

Make sure the indexes are present on the indexers as well as on the ES SH.