Does Splunk support IMDSv2 EC2 instances?

[jmatthews]

Hi

I've previously used imdsv1 on my EC2 instances to provide role credentials to allow my EC2 Splunk instance to reach across accounts to grab files.  I'm interested to find out if Splunk supports imdsv2 for credentials?  I haven't been able to find anything (nor get this to work).  

Thanks!

[terminaloutcome]

For future-folk finding this thread, IMDSv2 support was added in 8.1.5, the config's under the imds stanza in server.conf.

Set it like this, have an instance role for access to s3 and smartstore will Just Work™:

[imds]
imds_version = v2

[scannon4]

Anything new on this?  I am running Splunk 9.0.4 and I changed 2 of my indexers (EC2 instances) to IMDSv2 and then they started having some issues so I had to swap them back.

[phpguy_80]

any updates on this? does imdsv2 work yet?

[thambisetty]

No, Splunk doesn't support at least before or on 8.0.6

we observed that the below mentioned AWS service control policy (SCP) Sid was enforced at root organization level part of enhancing security and our current Splunk version is 8.0.6 which uses security credentials retrieved from IMDSv1.

We have performed below actions to confirm that Splunk uses security credentials retrieved from IMDSv1:

1. Added below SCP Sid to AWS account where our ec2 instances hosted and running Splunk. Splunk started logging s3 “access denied” errors
2. listed directories of the configured S3 bucket from one of the ec2 instances was success when we made s3api call from CLI
3. to confirm CLI by default used security credentials retrieved from IMDSv2, we configured keys retrieved from IMDSv1 in CLI environment variables and made same request as above but this time we observed access denied errors.
4. Reverted the change performed in step 1 then Splunk stopped logging S3 access denied errors, this confirmed that Splunk uses security credentials retrieved from IMDSv1.

[HiMAT-Mode]

I'm looking into this as well. From what I've been able to determine so far, The Splunk TA for AWS currently does not support IMDSv2 calls.

After installing the TA, you can find the logic it uses for discovering IAM roles located within Splunk_TA_aws/bin/splunk_ta_aws/common/aws_accesskeys.py. The logic does not attempt to retrieve a token from the IMDS API before making calls, thus non-compliant with IMDSv2. If the IMDSv1 call fails, the logic assumes that the app is not installed on an EC2 instance.

We will be opening a support case to address this.

[jbuckner85]

Hello HiMAT-Mode,

Did anything come of your case with Splunk on this? Our cloud engineering team has the same question regarding whether or not they will update the TA to support this.

Thank you!

[suzakutheknight]

Splunk_TA_aws/bin/splunk_ta_aws/common/aws_accesskeys.py has been updated to account for IDMSv2 as of Splunk Add-on for AWS 6.0.0. Look for fixed issue ADDON-47661 in the release notes: https://docs.splunk.com/Documentation/AddOns/released/AWS/Releasenotes.

I'm also going to mention a separate issue related to IMDSv2 that you might encounter if running SmartStore and relying on the EC2 instance profile to authenticate your S3 access. There is a setting in server.conf that you will need to enable on your indexers to allow IMDSv2 calls:

[imds]
imds_version = v2

[splunker686]

Hi, 
Were you able to confirm whether the latest Splunk Add-on for AWS supports IMDSv2?

Thanks.

[thambisetty]

@suzakutheknight @HiMAT-Mode 

appreciate if you post new question about AWS TA. The current question is all about ec2 and s3.