Why do I get the following messages in splunkd.log after installing Splunk Universal Forwarder in a GCP instance?

[Sithima]

Why do I get the following messages in splunkd.log after installing Splunk Universal Forwarder in a GCP instance?

12-16-2022 10:49:12.021 +0000 WARN AwsSDK [1903 ExecProcessor] - ClientConfiguration Retry Strategy will use the default max attempts.
12-16-2022 10:49:12.021 +0000 WARN AwsSDK [1903 ExecProcessor] - ClientConfiguration Retry Strategy will use the default max attempts.
12-16-2022 10:49:12.023 +0000 ERROR AwsSDK [1903 ExecProcessor] - EC2MetadataClient Http request to retrieve credentials failed with error code 404
12-16-2022 10:49:12.023 +0000 ERROR AwsSDK [1903 ExecProcessor] - EC2MetadataClient Can not retrive resource from http://169.254.169.254/latest/meta-data/placement/availability-zone

[alaprade36]

I also see the same ERRORS on a GCE instance. The only explanation for this is that AWS SDK is enabled out of the box and does not take into account CLOUD ENV where splunk is installed. In my mind or what should of been considered is that CLOUD SDK's can be enabled/disabled in server.conf or some other conf file. This is just sloppy if this in fact the case...unnecessary compute allocated to process irrelevant logging errors. 

[tomkreiner]

I see the same logs with a full Splunk Enterprise (currently 9.0.4) installation.

[burwell]

When I saw these in Splunk 9.0.1 I opened case 3093336.

Splunk's response is that AWSSDK will be disabled by default starting in version 9.1.0. AwsSDK errors are safe to ignore. Those messages are happening as part of the checks that were added to on-prem installation. I have requested an update to Splunk docs to properly reflect this. (Becky's note: I don't see this in known issues for 9.0.4)

As a short term workaround you can add "category.AwsSDK=FATAL" under the [splunkd] stanza in $SPLUNK_HOME/etc/log.cfg to silence the message.

I tested the above and it works but don't want the work to change the log.cfg as it changes with each version.

Note they did not give me a way to disable.