Splunk Enterprise Security: Why do correlation searches sometimes say "Access - " or "Network - " and sometimes not?

[proletariat99]

So if you create a new correlation search, a fancy little "feature" of Splunk Eenterprise Security, a stanza gets created in both correlationsearches.conf and also in savedsearches.conf... awesome. However, if you modify that correlation search using the gui or CLI, the correlationsearches.conf gets modified and the savedsearches.conf doesn't.

This is endlessly annoying.

However, when we start to see errors about stanzas missing from savedsearches.conf are even more annoying and are often the result of one .conf file having something like this

in $SPLUNK_HOME$/etc/apps/DA-ESS-AccessProtection/local

correlationsearches.conf

[Access - *search name here* - Rule]
... 
stuff 
...

savedsearches.conf

[Endpoint - *same search name here* - Rule]
...
stuff 
...

This stanza name mismatch sometimes happens right after creating a rule using the ES Config GUI. Sometimes it happens after modifying a search name in the GUI. Does anyone know how to avoid this stanza mismatch issue in a production ES implementation? We are trying to avoid forcing all correlation search engineers to use the CLI.

[LukeMurphey]

I recommend opening a bug report. If I understand the behavior correctly, then this is definitely not correct behavior. I very much would like to avoid having your engineers forced to use the CLI when ES has an editor specifically for this; we will definitely try to get this worked out.

Let me know when you open the ticket; I just want to know so that I can be ready in case this comes to the ES development team.

[dirkmeeuwsen]

I believe these are automatically concatenated when you first build a rule, through the GUI at least. There is a drop down for getting them in the correct domain so they show up correctly.

Also, something else I've found, we are a Cloud customer and thus do not have access to these conf files to modify or delete any correlation events. Something I am sure others will run into. Our "solution" to that is to create a test rule that we never change the name of, but modify the search in just to test rules. Once they are the way we want them, then we create a rule and put it into "production." So we just have a series of rules called "Username Test Rule" that automagically gets assigned to the creator so it doesn't cloud things up too much. You could also create some custom statuses to tune out of metrics if you are doing any on this data. Hope that helps in the meantime. Thanks!

[proletariat99]

and thus do not have access to these conf files to modify or delete any correlation events

OMG, that sounds miserable. That would be a deal-breaker right there. Sounds like your workaround is really creative and a good idea.

[dirkmeeuwsen]

Yeah, its been bittersweet. Nice to not have that level of responsibility, but the support for cloud could use some work, which is kind of a bummer. I know it will get better though.

[proletariat99]

agreed. It's the one constant in splunkland... there is always iteration in the right(ish) direction 🙂

[pj]

This is still an issue as of ES 4.1 and agree it is frustrating in a customer's cloud environment where there is no access to the backend configuration files. Additionally, you cant change the application context from the GUI once it is initially set, nor can you change the notable event security domain after the first save of the search (all greyed out). As a result, unless you remember to set the security domain when first saving, the search is hard coded to the default "Access" domain. If you then delete the correlation search from the Saved Searches gui to try to start again, you get errors every day from Splunk complaining about how savedsearches for X correlation search cant be located etc. Super frustrating.