Splunk Enterprise Security

Missing events in Notable Index Splunk Enterprise Security

courtneyj
Engager

Hello,

For your awareness my architecture consist of 1SH, 1 Enterprise Security SH,  Cluster of 3 indexes, deployment server with a cluster master, license master, and MC.

 I noticed there are no Notable Events being populated into my notable index. I created events that matched the correlation searches I turned on and also ran those CS searches in search separately to ensure it picked up the events I created. I validated the data models with pivot to ensure data was populating. I also tried to created a manual notable event and nothing showed up in Incident Review. Upon looking at the indexes in the setting menu I see a notable index but nothing is getting populated, likely because I am searching off my index cluster. My deployment server is only managing my core Splunk search head and the I read somewhere that the Splunk_SA_CIM app  needs to have a index.conf  for notable events to be place locally on ES. Can someone please provide some thoughts or suggestions. Thanks in advance..

0 Karma

lkutch_splunk
Splunk Employee
Splunk Employee

Here are some other tips for troubleshooting notable events: 

https://docs.splunk.com/Documentation/ES/6.4.1/Admin/Troubleshootnotables

courtneyj
Engager

Yeah I figured out what it was with a little assistance from Splunk Technical support. Since my ES search head was searching off a index cluster the notable index and all other ES required indexes in my index.conf were missing.  Once I added those indexes and applied the cluster bundle it worked like a charm. Also be advised if you go into your cluster master and don't see the indexes show right away that's because the indexes don't show up until something is written to them.

 

https://docs.splunk.com/Documentation/ES/6.4.1/Install/Indexes

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Shame on you for letting PS leave without verifying ES is working as it should.  Shame on PS for the same.


nothing is getting populated, likely because I am searching off my index cluster

All searches run off the indexer cluster.  If nothing is populated then it's because there's no data or because the search is faulty.

It's helpful for every SH to have a local copy of indexes.conf.  It probably won't solve this problem, but may help with some UI elements.

The notable index is only populated when correlation searches find events that meet their rules AND when the CS has an action to create a notable event.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...