CloudTrail Network Activity events not processed via Generic S3 input

[akkermansie]

We recently enabled Network Activity Events on our Organizational CloudTrail, and the logs are being delivered to an S3 bucket. Despite logs being delivered to S3, Splunk does not process or index these events, and queries return no results. 

The network activity logs are stored in the following path format: AWSLogs/<org_id>/<account_id>/CloudTrail-NetworkActivity/

We are using Splunk version 7.10.0 and don't see any errors in the logs.

[livehybrid]

Hi @akkermansie 

Have you set a 'Log File Prefix' in your Generic S3 input? 

I suspect its unrelated to your issue but version 8.1.0 is now available for the AWS Add-on so you may also want to consider upgrading.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[akkermansie]

We have not configured the log_file_prefix argument and the sourcetype is set to aws:cloudtrail.

I suspect that indexing CloudTrail events directly from an S3 bucket using the aws:cloudtrail sourcetype may only support CloudTrail Management Events. Other event types, such as Data Events or Network Activity events, might not be fully supported in this configuration. But I cannot find anything in the docs about this.

[richgalloway]

Have you configured and enabled an input as described in https://splunk.github.io/splunk-add-on-for-amazon-web-services/CloudTrail/ ?

P.S. It appears the version 7.10.0 is no longer available to consider upgrading to 7.11.0 or newer.

[akkermansie]

We are receiving and ingesting the Management events. As described in my first message, the issue is related to ingesting Network Activity.

https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-network-events-with-cloudtrail.ht...

[richgalloway]

You need an input to tell Splunk to ingest the Network Activity data.

[akkermansie]

Why we would need a new input for this? All of our logs are stored in a single S3 bucket (including Management events and Network activity events).

AWSLogs/<org_id>/<account_id>/CloudTrail/<logs>
AWSLogs/<org_id>/<account_id>/CloudTrail-NetworkActivity/<logs>

If we would create a new input, how would we make sure that input would only process Network Activity events?

[richgalloway]

That you are ingesting Management events, but not Network Activity events tells me there is an input defined for the former, but not the latter.  Perhaps this is not the intent, but it is the effect.

Can you share the CloudTrail input(s)?