Worked extensively with Splunk support on this. They believe that the problem is that the app is either fundamentally incompatible with Splunk 9 or the latest Salesforce TA.
Ultimately splunk-app-sfdc is using the collection lookup_sfdc_usernames_kvstore which is not defined in the collections.conf of the app, but in the add-on. It looks like the app is trying to refer to that and is not able to find that lookup.
ERROR KVStoreProvider [29936 SchedulerThread] - Could not create KvStore Lookup failed because collection 'lookup_sfdc_usernames_kvstore' in app 'splunk-app-sfdc' does not exist, or user 'splunk-system-user' does not have read access.
Hence my question: Has anyone gotten this to actually work? If so, what is the trick?
Regards.
Mike Kirda
Ultimately the dashboards do not work. When I look at the source, this is the query:
index=sfdc eventtype=sfdc-login-history Application=Browser |stats count by UserType| sort - count | eval label=UserType." (".count.")" (I added the index=sfdc to it as this is where the data resides.)
When in the main search and reporting context, I get results.
However when in the Salesforce app context, we see this error:
I do get results in the Search and reporting context, but when I run it in the Salesforce app context, I get the following @johnhuang:
Any help here from someone who has gotten it working would be highly appreciated.
Regards.
Mike Kirda
Glad I am not the only one who is having this problem. I've checked permissions all over the app & on the backend and have the same problem. Thanks for the heads up on the dashboards using the wrong index.
The lookup name is lookup_sfdc_usernames, not lookup_sfdc_usernames_kvstore.
1. Check if you have any data in the lookup:
| inputlookup lookup_sfdc_usernames
If 1 doesn't work:
Check if the above lookup in configured in lookup definition.
Check if the populating search "Lookup - USER_ID to USER_NAME" is scheduled.
Once you have the lookup working, you should modify the dashboards to point to this lookup.
At least for me there is no data from the lookup you indicated but to @mkirda1 's point, the error does point to lookup_sfdc_usernames_kvstore. I'll see if I can add a screenshot.
I do have the lookup definition there and it's enabled.
Aaron
Love to know if you get an answer to this as I'm having the same issue.