Why does modifying notable severity in Splunk ES impact historic events as well?

[vik_splunk]

Hi All,

We notice a seemingly weird behaviour where modifying the notable severity in a correlation search brings up historic events to the "Incident Review" pane with the new severity. 

We use Enterprise Security ver. 5.3.0

To explain further with a hypothetical scenario

1. Let's say A use case like "password violation on a critical asset" with a notable of informational severity fire 5 times a day on an average.
2. This noon, I change severity from informational to high
3. Navigating to the incident review and choosing a time period to 30 days(for instance) brings back 30 days worth of notables for this use case but with a high! severity (which is not right)
4. Ideally we expect the events to be split across two severities namely information until noon today and high for any events after

Anyone faced this issue? Is this by design? Is there a solution to this?

[thambisetty]

check below thread.

https://community.splunk.com/t5/Splunk-Enterprise-Security/ES-notable-urgency-changing-for-old-notab...

[vik_splunk]

Thanks! @thambisetty  appreciate the inputs

[Jhunter · Accepted solution]

I will take a stab - 

Doing a search on `notable` - I don't see the severity field in the raw notable logs. 

This lead me to believe that severity is probably in a lookup table somewhere. 

I found a lookup table correlationsearches_lookup that I found the severity field is stored in. It would make sense that when you change the severity in content management, all notable events would be changed. as the Urgency would be derived from this severity.

For a solution see this: 

https://docs.splunk.com/Documentation/ES/6.2.0/User/Howurgencyisassigned

You can manually calculate the severity in the SPL of the correlation search which according to that document: 

"Severity defined in the search syntax results in an event where severity takes precedence over the severity defined in the notable event adaptive response action."

So in your password violation scenario you could do something like (pseudo code):

eval rightNow=now()   [get running time stamp]

eval severity=case(rightNow later than noon, "high", rightNow earlier than noon, "informational)

[Laszlo_K]

Another solution may be to add this (or a macro) at the end of the rule to get the severity from the lookup based on the search name:

| lookup correlationsearches_lookup rule_name as $search_name$ OUTPUTNEW severity

[vik_splunk]

Thanks @Jhunter  appreciate the inputs

[vik_splunk]

Ideas if any anyone? Kindly respond. Thanks!