I'm trying to use the Microsoft Azure Add-on for Splunk and was successful in getting this add-on to ingest Azure AD User data via the supplied input. When trying to use the Azure AD Sign-in input; I'm not getting any data and I'm seeing the following error when looking in the logs.
index="_internal" host=xxxx source="/opt/splunk/var/log/splunk/ta_ms_aad_MS_AAD_signins.log"
2020-04-24 15:07:53,551 ERROR pid=19474 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 127, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_signins.py", line 84, in collect_events input_module.collect_events(self, ew) File "/opt/splunk/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_signins.py", line 62, in collect_events query_date = get_start_date(helper, check_point_key) File "/opt/splunk/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_signins.py", line 37, in get_start_date d = helper.get_check_point(check_point_key) File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 518, in get_check_point self._init_ckpt() File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 509, in _init_ckpt scheme=dscheme, host=dhost, port=dport) File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/modular_input/checkpointer.py", line 166, in __init__ scheme, host, port, **context) File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/solnlib/utils.py", line 167, in wrapper raise last_ex HTTPError: HTTP 402 Payment Required -- Requires license feature='KVStore'
About this setup: The add-on is running on a Heavy Forwarder and this forwarder is in the forwarder license group; forwarding to Splunk Cloud. I've double checked all the permissions that the registered app needs in Azure and I think I'm good there. This same registered app is in use with the legacy Microsoft Azure Active Directory Add-on to pull sign-in and audit logs today. The permissions I've granted the registered app are here:
Thoughts on what may be going on here?
Thanks! I had seen that post, but Splunk support did not want to provide me with the 0GB/day license that enables KV Store. They kept telling to contact the developer of the Add-on and that they didn't support it. That using the Forwarder Group license should be all I need.
I did, in my haste, try the Free License and that worked. Finally support suggested I copy the enterprise 0GB/day license they provided 2 years ago on another heavy forwarder to this one and that worked too (I should have thought about that before them).
Not sure why this add-on is not working with the normal Forwarder Group License on a Heavy Forwarder, I feel like it should.