The answer by mdessus describes how to detect this issue.
Firstly, the ES Identity and Assets are merged every 5 minutes as a modular input, that explains why sometimes it will happen instantly and other times it can take a few minutes: http://docs.splunk.com/Documentation/ES/4.1.1/User/Identitymanagement#Merging_the_asset_and_identity_lists
What worked for me was the following:
Background: you have a lookup, ad_identity_list that is silently failing to load in to ES. The lookup is populated with good data, you've checked the logs for modular inputs and have seen that the merge is running properly, but no data Identity data is being populated in ES.
Make an interim lookup, called something like ad_identity_interim.
Copy whole ad_identity_list into ad_identity_interim.
Execute the following, to place only a few entries into the Identity lookup ES is trying to merge.
| inputlookup ad_identity_interim | head 5 | outputlookup ad_identity_list
Wait until the merge occurs and you should see the five entries in your Identity Center.
Continue adding incrementally until you have the whole list in there, making sure you wait for the merge to occur between each execution.
| inputlookup ad_identity_interim | head 50 | outputlookup ad_identity_list
| inputlookup ad_identity_interim | head 100 | outputlookup ad_identity_list
| inputlookup ad_identity_interim | head 500 | outputlookup ad_identity_list
| inputlookup ad_identity_interim | head 1000 | outputlookup ad_identity_list
You should now have all your identities in ES.
I'm unsure as to why this works, but the issue has occurred and this fix has worked for me in several completely different architectures. It seems as though once the initial list has populated that the updates to the lookup are loaded properly, so I haven't had to make a chain of saved searches to behave as described above; it works as expected once it's all initially loaded -- noting that I have only ever made minor changes on an ongoing basis.
... View more