topic Re: Error Ingest Data AWS Cloudtrail "error=Traceback (most recent call last):" in Deployment Architecture

Error Ingest Data AWS Cloudtrail "error=Traceback (most recent call last):"

zksvc — Wed, 11 Jun 2025 11:01:48 GMT

Hi Everyone,

I encountered an error while ingesting sourcetype=aws:cloudtrails in AWS Apps. I attempted to ingest data from the following sources: aws:waflogs, aws:network-firewall-log, aws:cloudtrails, aws:securityhub-log-group. However, upon checking, only aws:waflogs and aws:network-firewall-log were ingested. Attached below are the errors from the logs.

Also i screenshot inputs config from the apps side here :

Last i show you the proof if i only received that 2 sourctypes here :

If you have any experience from this issue, please give me the answer.

Danke,

Zake

Re: Error Ingest Data AWS Cloudtrail "error=Traceback (most recent call last):"

livehybrid — Wed, 11 Jun 2025 11:09:25 GMT

Hi @zksvc

It looks like the inputs are polling AWS Cloudwatch too frequently, which is giving your Rate Limit exception.

If you have just set this up then it will be trying to pull logs back from whatever the only_after date you set was (see https://splunk.github.io/splunk-add-on-for-amazon-web-services/CloudWatchLogs/ for input config descriptions)

If you left this field blank then I believe it tries to load all the events in the Cloudwatch logs group in AWS. Ultimately it looks like its repeatedly querying CW Logs to get more logs which is why it is hitting the rate limit. The number of polls to CW Logs will reduce once it has caught up to the current date. It might be worth enabling one at a time to allow them to catch up gradually.

If you do not need the historic data then I would suggest cloning the inputs and setting the only_after date to a recent date and then deleting the old input. I dont think it is possible to change the only_after once created because of how the checkpoint of the current date/time is recorded, but I may be wrong here.

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Error Ingest Data AWS Cloudtrail "error=Traceback (most recent call last):"

zksvc — Wed, 11 Jun 2025 11:16:13 GMT

Thanks for your reply,
I will try to change the interval time to 600 seconds first.

Re: Error Ingest Data AWS Cloudtrail "error=Traceback (most recent call last):"

zksvc — Thu, 12 Jun 2025 01:51:41 GMT

Hi @livehybrid I have changed the interval to 600 seconds, but the data is still not available. Is there any other solution that you know?