All Apps and Add-ons
Highlighted

Splunk TA for AWS "Content Has Been Modified"

Splunk Employee
Splunk Employee

We are using SplunkTAAWS 4.6.0. On an EC2 instance with a proper IAM instance profile which has access to SQS and S3, enable an SQS-based-S3 input on 1GB+ S3 keys.
The modular input has a watch path on splunktaaws/settings/account/YOURINSTANCEPROFILE and exits on change, even though the credentials are still valid for +6 hours. The number of messages in "in flight" grow as modular inputs keep resetting due to the credential changes. The auto-discovered instance profile role credentials cause modular input to exit prematurely. This causes SQS messages to remain in flight. The mod input gets re-ingested and causing duplicate data.

Log messages below;
2019-03-03 14:49:58,504 level=INFO pid=16987 tid=MainThread logger=splunksdc.config pos=config.py:check:70 | starttime=1551624493 datainput="BibliosSQSQueueChronicleExecveTTYIAD" | message="Content has been modified." path="splunktaaws/settings/account/BibliosIAMRoleSplunk"
2019-03-03 14:50:00,221 level=INFO pid=16987 tid=MainThread logger=splunksdc.collector pos=collector.py:run:248 | | message="Modular input exited."

Highlighted

Re: Splunk TA for AWS "Content Has Been Modified"

Splunk Employee
Splunk Employee

We have a plan to improve the behaviour of the modInput which will be done on version 5.
Until then you can consider to tune it by modifying the timer value as below;

---- file: etc/apps/SplunkTAaws/bin/splunksdc/config.py

74 def hasexpired(self):
75 now = time.time()
76 if now - self.
lastcheck > 30:
### This is 30seconds timer you may want to try like, 20000 considering the previous credentials is valid for the next 6 hours ( 6 X 3600) ###
77 eelf.
lastcheck = now
78 self.
hasexpired = self.check()
---- file: etc/apps/SplunkTAaws/bin/splunksdc/config.py

This has reduced the chances of duplicate data by 95+ %.

View solution in original post