After a few months of back and forth with Splunk Support, we got to the root cause. The credit for this goes to the unnamed (sorry, I don't know their name) AWS Add-on developer, who the support case got escalated to. They flagged that there was a global boto configuration file (/etc/boto.cfg) which was causing a conflict with some assumed defaults which the AWS Add-on relies on. To summarise: Problem: - This problem presented as a hanging / spinning ‘Loading’ icon in the Splunk AWS Add-on Inputs and Configuration views. - A search in the _internal index for “aws ERROR” showed python stack trace error messages. - The problem appears to be configuration in the /etc/boto.cfg file. This is a global configuration file, so the settings in here are conflicting with the defaults (if there was no config file), which the Splunk AWS Add-on relies on. - The file is created due to the default installation of Google’s gcloud / gsutil command line tools. The tools are installed by default on Google Cloud Compute (GCP) hosted servers. - Our initial AWS integration testing was using GCP account to host the HFs, whilst waiting for access to a different AWS account. Solution: - You don't really want remove the /etc/boto.cfg configuration file because those servers might need these tools to access GCP services / APIs. - In addition, even if we removed it, there is the possibility that other tools could create it in the future. - Therefore the fix is to do something which will only affect Splunk. - What we did was, in the $SPLUNK_HOME/etc/splunk-launch.conf file we added the environment variable: BOTO_CONFIG=/tmp/does_not_exist - This sets the BOTO_CONFIG variable for just the Splunk process specifically to a file which doesn’t exist. - Because the files doesn’t exist, the defaults, as opposed to the global /etc/boto.cfg are used. - Therefore there is now no conflict for the Splunk AWS Add-on, which runs under the Splunk process. I hope this helps others who may be having similar issues. Or at least points them in a direction to look into. Graham. ... View more