Splunk Enterprise Security

Enterprise Security Identity Correlation - merging of identities

DMohn
Motivator

Hi all,

I have a problem understanding how ES Identity Correlation merges identities together.

Example:
I have a LDAP lookup A that gets me some users with identities like username1|email.of@user.company - as well as another LDAP Lookup that gets identities like username_1a|email.of@user.company. My expectation would be that ES merges the user that I can search the Identity Center for username1 and I will see username_1a as well - as they have the common identification of email.of@user.company. However, this does not work.

Can someone enlight me why this is the case? Do I miss something here? I thought of merging both LDAP lookups together and outputting only one identity file. This however would spoil the possibility of giving username_1a another priority, as these are admin accounts.

Any suggestions?

0 Karma
1 Solution

nickhills
Ultra Champion

In my experience, we got a lot of value building our own identities dataset which is then exposed to ES as a single csv file.

It is more work, but I combine user details from LDAP/our ticketing system/HR databases etc to give a very comprehensive identity lookup (it actually contains more fields than ES requires), and we include users main accounts, and their admin accounts, as well as proxy smtp addresses, so regardless of who/how they login they get resolved to the right person etc.
We run a number of jobs which pull data from various systems, and write them into a summary index, and then every 4 hours combine the summarized data into the CSV

If my comment helps, please give it a thumbs up!

View solution in original post

nickhills
Ultra Champion

In my experience, we got a lot of value building our own identities dataset which is then exposed to ES as a single csv file.

It is more work, but I combine user details from LDAP/our ticketing system/HR databases etc to give a very comprehensive identity lookup (it actually contains more fields than ES requires), and we include users main accounts, and their admin accounts, as well as proxy smtp addresses, so regardless of who/how they login they get resolved to the right person etc.
We run a number of jobs which pull data from various systems, and write them into a summary index, and then every 4 hours combine the summarized data into the CSV

If my comment helps, please give it a thumbs up!

DMohn
Motivator

This was the solution I was thinking of as well. But how do I solve the problem with different priorities? Do you have a something like this in place?

0 Karma

nickhills
Ultra Champion

We do.
Without giving too much away, we use a PAM access management system which keeps user-admin accounts separate from real-admin accounts. These get marked highest priority, then system/service accounts get classified based on their access levels, and 'real' users get marked differently again.

I am being a bit guarded with what I say, but in my environment, an Identity only has priority, even though it can leverage other identities (with different priorities)

Not sure if that makes sense, but hit me up on Slack (same username) and I can explain a bit better

If my comment helps, please give it a thumbs up!
0 Karma

lakshman239
SplunkTrust
SplunkTrust

As far as I know, your understanding is correct, ES merges them, if you define them in the macros.conf under SA-IdentityManagement app.

You would need to add both your lookups (append) in the local/macros.conf identity_sources macro and this will then produce consolidated (identities expanded.csv) having data merged to one lookup that ES uses in dashboards and in other macros/lookups.

0 Karma

DMohn
Motivator

This is what I would expect, but it seems not to work that way. I have the lookups defined as identity lookups in ES Config, and they do get merged by the IdentityManagement macro. Still, I am not able to correlate between the two different user names. I would expect the Identity center to show me both identites of the user if I search for username1 in my example. This is not the case. But if i search for the email address, which is the same for both identites, i get both of them.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

what do you see when you run the below ?

|makeresults | eval user="yourname" |get_identity4events(user)`

when the two files merge, in the identities, you would be able to see "|" separated values.

I assume your scheduled merge process works, else you may have to run the gensearches manually!?

0 Karma

DMohn
Motivator

I see 1 result line, with username and email address of this user. But not his second identity.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

I believe if you are having identity 1 and identity 2 for same user, you would need to have them merged (using |pipe) in one of the identity lookup, so a single asset/identity_id is assigned as part of merge. Not sure if ES looks at two diff lookups to match 'key' and form one identity.

0 Karma

nickhills
Ultra Champion

Correct.
That’s why I include email in the identity field of the csv. If there is a common identity in two sources they should merge, but you need them to share an email address in the id field.
Since ad won’t allow two users to share an email, this is why I build my own logic to do all the merging and then give ES a mostly merged csv with all the heavy lifting complete.

If my comment helps, please give it a thumbs up!
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...