After Upgrading Splunk Enterprise Security, I am not receiving Incidents

[Stefanie]

Hey!

We upgraded to Splunk Enterprise Security to the latest version a few weeks ago.

Before, it was on Version 4.x I believe. It was detecting events before we upgraded and after the upgrade, no more events.

Some of the events that were being triggered were:

Brute Force Access Behavior Detected, Geographically Improbable Access Detected, and Threat Activity Detected.

Where can I go to find out why I am not detecting these items after the upgrade? The objects for it are enabled in the Content Management Screen.

I will copy and paste what it says below:

Statistics

Avg. Event Count 0 Avg. Result Count 0 Avg. Run Time 0:00:01 Invocations 24 Skipped 0 Success 24 Update Time Nov 19, 2021 1:00:00 PM

The data models have a green checkmark and so do the lookups.

[ro_mc]

First, are you using Splunk Enterprise version 8.x?
Earlier versions are not compatible with the latest ES version of 6.6.x:

https://docs.splunk.com/Documentation/VersionCompatibility/current/Matrix/CompatMatrix

Next, check the post-upgrade documentation to confirm you've completed all the required steps to have ES working correctly. Also verify that your previous notable events are present, as these may need to be restored from your KV store backup given the large gap between Splunk ES versions.

Create a manual notable event, and verify that this appears in the incident review page. This will ensure that Enterprise Security is (for the most part) configured correctly.

If required, check index=_internal sourcetype=splunkd source=*splunkd.log " ERROR " OR " WARN ". You may want to check other log sources as well, but this will point to the most significant problems.

Confirm that correlation searches have run, and verify that they are generating events that result in creation of notables. The upgrade from 4.x to 6.x is quite significant and much may have changed in the SPL of these alerts. E.g. they may have referred to indexes and sourcetypes previously, but now refer to datamodels. If necessary, create your own enabled correlation search with a notable event action and verify that this appears on the incident review page.

If the searches are running, and the notable event index is populating, but incident review is empty, check to see if notable events are being created in index=notable, and if so, check the `notable` and `incident_review` macros to determine if any errors are occurring during enrichment. This could indicate a problem with the KV store lookups or the underlying MongoD service that it runs on.

Let us know how you go.