All Apps and Add-ons

splunk add on for aws doesnot report cloudwatch logs


We are trying to ingest cloudwatch logs to splunk using splunk add-on for AWS. Some of the logs appear fine but there is a delay of more than 1 hour. The splunk server and forwarder are in the same time zone. And some of the logs dont even appear. Below is the error we are geting:

2018-05-18 17:14:32,803 level=ERROR pid=4348 tid=Thread-4 logger=splunk_ta_aws.modinputs.cloudwatch_logs.aws_cloudwatch_logs_data_loader | | message="Failure in describing cloudwatch logs streams due to throttling exception for log_group=app1/container, sleep=2.5481909735, reason=Traceback (most recent call last):
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/cloudwatch_logs/", line 63, in describe_cloudwatch_log_streams
group_name, next_token=buf["nextToken"])
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/boto/logs/", line 308, in describe_log_streams
File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/boto/logs/", line 576, in make_request
JSONResponseError: JSONResponseError: 400 Bad Request
{u'__type': u'ThrottlingException', u'message': u'Rate exceeded'}

Any help or suggestions are appreciated

0 Karma

Path Finder
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.