Splunk TA for AWS "Content Has Been Modified"

[sylim_splunk]

We are using Splunk_TA_AWS 4.6.0. On an EC2 instance with a proper IAM instance profile which has access to SQS and S3, enable an SQS-based-S3 input on 1GB+ S3 keys.
The modular input has a watch path on splunk_ta_aws/settings/account/YOURINSTANCEPROFILE and exits on change, even though the credentials are still valid for +6 hours. The number of messages in "in flight" grow as modular inputs keep resetting due to the credential changes. The auto-discovered instance profile role credentials cause modular input to exit prematurely. This causes SQS messages to remain in flight. The mod input gets re-ingested and causing duplicate data.

Log messages below;
2019-03-03 14:49:58,504 level=INFO pid=16987 tid=MainThread logger=splunksdc.config pos=config.py:_check:70 | start_time=1551624493 datainput="BibliosSQSQueueChronicleExecveTTYIAD" | message="Content has been modified." path="splunk_ta_aws/settings/account/BibliosIAMRoleSplunk"
2019-03-03 14:50:00,221 level=INFO pid=16987 tid=MainThread logger=splunksdc.collector pos=collector.py:run:248 | | message="Modular input exited."

[sylim_splunk · Accepted solution]

We have a plan to improve the behaviour of the modInput which will be done on version 5.
Until then you can consider to tune it by modifying the timer value as below;

---- file: etc/apps/Splunk_TA_aws/bin/splunksdc/config.py

74 def has_expired(self):
75 now = time.time()
76 if now - self._last_check > 30:
### This is 30seconds timer you may want to try like, 20000 considering the previous credentials is valid for the next 6 hours ( 6 X 3600) ###
77 eelf._last_check = now
78 self._has_expired = self._check()
---- file: etc/apps/Splunk_TA_aws/bin/splunksdc/config.py

This has reduced the chances of duplicate data by 95+ %.

[salexander8]

Hi,

We have the same issue and are using version 5.01. Looking at the source code, this has not changed in this version or the latest version 5.03. Can you please advise what the latest fix is. We are using a managed cloud instance so can't change any of the source code.

Thanks

Regards