All Apps and Add-ons

Why is the Splunk Add-on for Amazon Web Services not working and discarding events?

sean_kcg
New Member

I have the aws configured with the app. I am able to pull from some buckets, but not others. I see the following in the logs. I am not sure why it is discarding events. Any ideas?

NFO pid=22464 tid=MainThread file=aws_cloudtrail.py:process_CT_notifications:634 | fetched 520 records, wrote 0, discarded 520, redirected 0 from s3:aws-hhs-cms-oc-cloudtrail/AWSLogs/.../CloudTrail/us-east-1/2015/05/28/..._CloudTrail_us-east-1_20150528T0150Z_arraztklBqzeCN6r.json.gz
2015-05-27 21:53:53,814 INFO pid=22464 tid=MainThread file=aws_cloudtrail.py:stream_events:328 | 1 completed, 0 failed while processing a notification batch of 1 [0 errors deleting 1 notifications]  Elapsed: 0.981s
0 Karma

jacobwilkins
Communicator

Look at your inputs.conf stanza for your clouldtrail input.

Does it have this:

exclude_describe_events = 1

That is the default. If so, then what you are seeing is expected behavior. You'll pull down, parse, and discard all your "describe" events. You'll only index events that describe changes.

zoeygirl45
Explorer

Yes, my settings are set to 1, is this a recommended setting?

0 Karma

zoeygirl45
Explorer

Thank you, I will check and post findings.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.