Splunk Add-on for AWS is not loading Proxy Settings to connect to AWS SQS Queue

augrw — Wed, 02 Sep 2020 08:06:54 GMT

Hi,

I am using the Splunk Add-on for AWS app (5.01) to ingest data from SQS/S3 to Splunk 8.0.3 on-prem.

Our network gateway requirements is that the traffic is going through a proxy.

For some reason, this add-on is not able to communicate with proxy setting to establish connection to AWS SQS queue. It's keep hitting the below errors:

level=ERROR pid=14252 tid=MainThread logger=splunk_ta_aws.modinputs.sqs.aws_sqs_data_loader pos=aws_sqs_data_loader.py:log:93 | | message="Failed to get SQS queue url"

..............................

Files\\Splunk\\etc\\apps\\Splunk_TA_aws\\bin\\3rdparty\\python3\\botocore\\httpsession.py\", line 283, in send\n raise EndpointConnectionError(endpoint_url=request.url, error=e)\nbotocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: \"https://sts.amazonaws.com/\"\n"

==========

However, I am able to get SQS queue URL when I use AWS CLI in windows power-shell and CMD respectively:

PS C:\> aws sqs get-queue-url --queue-name C*******-Sqs
{
"QueueUrl": "https://sqs.ap-southeast-2.amazonaws.com/1*************/C***********-Sqs"
}

I noticed from the Splunk Add-on for AWS web UI there are proxy settings. When set, it generates a passwords.conf file containing a hash - I am guessing that it is the Proxy URL hashed.

As we do everything via configuration management, it would be awesome to just have this in the conf file.

I did notice that in Splunk Add-on for AWS's aws_settings.conf file, there is a Proxy stanza. When I set the options in this it doesn't take effect in the Web UI.

Where can I find the proxy conf file for this addon? How can I identify whether this addon is using my proxy settings at all?

I am stuck with this issue for days, and please help!

Re: Splunk Add-on for AWS is not loading Proxy Settings to connect to AWS SQS Queue

dwest_splunk — Wed, 06 Jan 2021 15:34:25 GMT

Hi
I think there was a UI bug in version 5.0.1 of the Addon, so best use the 5.0.3 version.

Or you may be able to update by upgrading httplib2 in bin/3rdparty/python/httplib2 but I think upgrade to 5.0.3 is the easier option.

Re: Splunk Add-on for AWS is not loading Proxy Settings to connect to AWS SQS Queue

sososplunks — Fri, 19 Mar 2021 16:32:14 GMT

Hi @augrw . Were you ever able to solve this issue. I'm currently facing it at the moment.

topic Re: Splunk Add-on for AWS is not loading Proxy Settings to connect to AWS SQS Queue in All Apps and Add-ons

Splunk Add-on for AWS is not loading Proxy Settings to connect to AWS SQS Queue

Re: Splunk Add-on for AWS is not loading Proxy Settings to connect to AWS SQS Queue

Re: Splunk Add-on for AWS is not loading Proxy Settings to connect to AWS SQS Queue