topic Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working? in All Apps and Add-ons

Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

Log_wrangler — Tue, 29 Sep 2020 23:56:22 GMT

I have a number of generic s3 inputs configured and indexing - normally without issue.
I can see in the logs for the working inputs show indexing s3 data is completing.

When I look at the newly created input, I see the same log messages, EXCEPT - indexed s3 data.

message="Start processing."
message="Start processing" last_modified="2019-03-01T00:00:00.000Z" latest_scanned="2019-04-02T21:05:31.000Z"
message="Start of discovering S3 keys."
message="begin loading credentials"
message="load credentials succeed"
message="Create new S3 connection."
message="End of fetching S3 objects."
message="Sweep ckpt file after completion of key discovering."
message="End of processing!"
message="The last data ingestion iteration hasn't been completed yet."

but there is NO message="Indexed S3 files." Like I see with the successful aws inputs. ... and there is no s3 data for that input coming in.

aws add-on is 4.4.0 on Splunk 6.4.1 HF

Can anyone point me in the right direction?

Please advise.

Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

chans28 — Sat, 15 Jun 2019 16:42:16 GMT

Did you ever figure out why this was happening? I have the exact same issue.

Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

suhasinihulikal — Mon, 05 Aug 2019 14:52:25 GMT

Did you figure out the issue behind this? I am stuck with the same issue.

Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

chans28 — Wed, 30 Sep 2020 01:36:36 GMT

For us it turned out to be the the AWS TA has 4 cacert files that need updating if your companies network has their own SSL certs. 3 of 4 are named conventionally as cacert.pem. However, 1 is named cacert.txt in this
directory

/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/boto/cacerts/cacerts.txt

Once we updated that with our company's certs everything started working.

Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

abhiravi7 — Tue, 06 Aug 2019 13:48:39 GMT

Did you figure out the issue behind this? I am stuck with the same issue.

Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

kagamalai — Mon, 21 Jun 2021 10:12:49 GMT

I am also facing exact same issue

| message="The last data ingestion iteration hasn't been completed yet."

Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

xiyangyang — Wed, 30 Jun 2021 06:12:32 GMT

I am also facing exact same issue

Is there any progress about this issue?

Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

kagamalai — Mon, 05 Jul 2021 05:57:44 GMT

There is no improvement, but every time changing the configuration and restarted the Splunk service it will take an hour to resume the logs flow