All Apps and Add-ons

Splunk App for AWS: Why the S3 buckets list does not show in the Splunk Add-on

xiyangyang
Path Finder

I can get cloudwatch log and description successfull but not S3 buckets.

My splunk EC2 has full access right to the s3 buckets but they are not listed in the Add-on setting page.

I tried to get the s3 buckets from Add data, but I got internal logs like:

"The last data ingestion itertion hasn't been completed yet"

I attcheched the internal logs, hope soemone can help.

---------------internal logs----------------------------------------------2021-01-18 18:57:59,217 level=ERROR pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:index_data:91 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="Failed to collect data through generic S3." start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066"
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 86, in index_data
    self._do_index_data()
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 107, in _do_index_data
    self.collect_data()
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 153, in collect_data
    self._discover_keys(index_store)
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 224, in _discover_keys
    bucket = self._get_bucket(credentials)
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 370, in _get_bucket
    bucket = conn.get_bucket(self._config[asc.bucket_name])
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/connection.py", line 509, in get_bucket
    return self.head_bucket(bucket_name, headers=headers)
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/connection.py", line 528, in head_bucket
    response = self.make_request('HEAD', bucket_name, headers=headers)
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/common/boto2_s3_patch.py", line 12, in wrapper
    response = func(*args, **kwargs)
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/connection.py", line 671, in make_request
    retry_handler=retry_handler
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/connection.py", line 1084, in make_request
    retry_handler=retry_handler)
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/connection.py", line 1043, in _mexe
    raise ex
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/connection.py", line 956, in _mexe
    request.body, request.headers)
  File "/opt/splunk/lib/python3.7/http/client.py", line 1244, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/splunk/lib/python3.7/http/client.py", line 1290, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/splunk/lib/python3.7/http/client.py", line 1239, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/splunk/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/opt/splunk/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/https_connection.py", line 119, in connect
    sock = socket.create_connection((self.host, self.port), self.timeout)
  File "/opt/splunk/lib/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/opt/splunk/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
socket.timeout: timed out

2021-01-18 18:57:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:57:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:56:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:56:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:55:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:55:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:54:53,444 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:54:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:53:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:53:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:52:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:52:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:51:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:51:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:50:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:50:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:49:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:49:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet."
2021-01-18 18:48:53,545 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_get_bucket:365 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="Create new S3 connection."
2021-01-18 18:48:53,545 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.common.aws_credentials pos=aws_credentials.py:load:163 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="load credentials succeed" arn="arn:aws:sts::052086164386:assumed-role/splunk_ec2_access/i-000c869a7adf815b0" expiration="2021-01-18 15:23:20+00:00"
2021-01-18 18:48:53,444 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.common.aws_credentials pos=aws_credentials.py:_load_source_credentials:195 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="fetch ec2 instance credentials"
2021-01-18 18:48:53,444 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.common.aws_credentials pos=aws_credentials.py:load:156 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="begin loading credentials" aws_account="splunk_ec2_access" aws_iam_role=None
2021-01-18 18:48:53,444 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_discover_keys:220 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="Start of discovering S3 keys."
2021-01-18 18:48:53,443 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:collect_data:143 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066" | message="Start processing" last_modified="2020-12-17T07:30:59.000Z" latest_scanned=""
2021-01-18 18:48:53,442 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:105 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066" | message="Start processing."

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...