All Apps and Add-ons

Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

Log_wrangler
Builder

I have a number of generic s3 inputs configured and indexing - normally without issue.
I can see in the logs for the working inputs show indexing s3 data is completing.

When I look at the newly created input, I see the same log messages, EXCEPT - indexed s3 data.

message="Start processing."
message="Start processing" last_modified="2019-03-01T00:00:00.000Z" latest_scanned="2019-04-02T21:05:31.000Z"
message="Start of discovering S3 keys."
message="begin loading credentials"
message="load credentials succeed"
message="Create new S3 connection."
message="End of fetching S3 objects."
message="Sweep ckpt file after completion of key discovering."
message="End of processing!"
message="The last data ingestion iteration hasn't been completed yet."

but there is NO message="Indexed S3 files." Like I see with the successful aws inputs. ... and there is no s3 data for that input coming in.

aws add-on is 4.4.0 on Splunk 6.4.1 HF

Can anyone point me in the right direction?

Please advise.

0 Karma

abhiravi7
Engager

Did you figure out the issue behind this? I am stuck with the same issue.

0 Karma

chans28
Explorer

For us it turned out to be the the AWS TA has 4 cacert files that need updating if your companies network has their own SSL certs. 3 of 4 are named conventionally as cacert.pem. However, 1 is named cacert.txt in this
directory

/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/boto/cacerts/cacerts.txt

Once we updated that with our company's certs everything started working.

0 Karma

suhasinihulikal
Explorer

Did you figure out the issue behind this? I am stuck with the same issue.

chans28
Explorer

Did you ever figure out why this was happening? I have the exact same issue.

kagamalai
Explorer

I am also facing exact same issue 

| message="The last data ingestion iteration hasn't been completed yet."

0 Karma

xiyangyang
Path Finder

I am also facing exact same issue 

Is there any progress about this issue?

0 Karma

kagamalai
Explorer

There is no improvement, but every time changing the configuration and restarted the Splunk service it will take an hour to resume the logs flow 

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...