We'd like to use the SQS based S3 method in the Splunk Add-on for AWS, but have trouble connecting to the endpoint. From my experience it only tries to connect to the legacy URL, which is not supported with the SQS endpoint:
"Private DNS doesn't support legacy endpoints such as queue.amazonaws.com or us-east-2.queue.amazonaws.com." source
Legacy URL: REGION.queue.amazonaws.com
New URL: sqs.REGION.amazonaws.com.
I was able to create a Band-Aid fix by adding the legacy URL to /etc/hosts with the new URL's IP, but that's a fragile solution and doesn't support multiple AZs.
In testing this I found that the AWS CLI supports redirecting through the use of a --endpoint-url argument:
aws sqs receive-message --endpoint-url https://REGION.amazonaws.com/ --queue-url https://sqs.REGION.amazonaws.com/ACCT#/QUEUENAME
It also looks like S3 might support setting a host_name value in the inputs.conf file to allow it to connect to a different endpoint.
Is there a method for setting the endpoint URL for SQS?
Thank you,
Erocky
When you've added your SQS input it should set the sqs_queue_url field, something like:
[aws_sqs_based_s3://YourInputName]
aws_account = loader
aws_iam_role = audit_cloudtrail
index = my_index
interval = 300
s3_file_decoder = ELBAccessLogs
sourcetype = aws:elb:accesslogs
sqs_batch_size = 10
sqs_queue_region = eu-west-2
sqs_queue_url = https://eu-west-2.queue.amazonaws.com/<yourAccountID>/<yourSQSQueue>
You should then be able to update the queue URL to
sqs_queue_url = https://sqs.REGION.amazonaws.com/<YourAccountID>/<YourQueueNamee>