All Apps and Add-ons

Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

Log_wrangler
Builder

I have a number of generic s3 inputs configured and indexing - normally without issue.
I can see in the logs for the working inputs show indexing s3 data is completing.

When I look at the newly created input, I see the same log messages, EXCEPT - indexed s3 data.

message="Start processing."
message="Start processing" last_modified="2019-03-01T00:00:00.000Z" latest_scanned="2019-04-02T21:05:31.000Z"
message="Start of discovering S3 keys."
message="begin loading credentials"
message="load credentials succeed"
message="Create new S3 connection."
message="End of fetching S3 objects."
message="Sweep ckpt file after completion of key discovering."
message="End of processing!"
message="The last data ingestion iteration hasn't been completed yet."

but there is NO message="Indexed S3 files." Like I see with the successful aws inputs. ... and there is no s3 data for that input coming in.

aws add-on is 4.4.0 on Splunk 6.4.1 HF

Can anyone point me in the right direction?

Please advise.

0 Karma

abhiravi7
Engager

Did you figure out the issue behind this? I am stuck with the same issue.

0 Karma

chans28
Explorer

For us it turned out to be the the AWS TA has 4 cacert files that need updating if your companies network has their own SSL certs. 3 of 4 are named conventionally as cacert.pem. However, 1 is named cacert.txt in this
directory

/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/boto/cacerts/cacerts.txt

Once we updated that with our company's certs everything started working.

0 Karma

suhasinihulikal
Explorer

Did you figure out the issue behind this? I am stuck with the same issue.

chans28
Explorer

Did you ever figure out why this was happening? I have the exact same issue.

kagamalai
Explorer

I am also facing exact same issue 

| message="The last data ingestion iteration hasn't been completed yet."

0 Karma

xiyangyang
Path Finder

I am also facing exact same issue 

Is there any progress about this issue?

0 Karma

kagamalai
Explorer

There is no improvement, but every time changing the configuration and restarted the Splunk service it will take an hour to resume the logs flow 

0 Karma
Get Updates on the Splunk Community!

How I instrumented a Rust application without knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...