All Apps and Add-ons

Configuring AWS S3 data input with the exact path

denismo
New Member

I configured the AWS S3 input data source in Splunk Cloud. After specifying the bucket, it presented a choice of paths which are only 2 levels down from the root. Our logs are stored in a location which is 3 levels down, and at 2 levels down we have our data files and logs. So specifying the 2 level folder will ingest all our data and logs which is obviously not desirable.

Is there a way to either specify the exact path from which the logs should be read, or making the configuration UI look 3 levels down?

Thanks.

Denis

Tags (1)
0 Karma

_d_
Splunk Employee
Splunk Employee

Denis, there are two ways to work with S3 data "paths" not available in the dropdown for now: use of blacklists/whitelists or your modify the key_name attribute (i.e. "path") in the inputs.conf file. Also, note that S3 is unlike other filesystems where there is no "hierarchy" per se and each "path" is really a key name that identifies a file/object in your bucket.

0 Karma
Get Updates on the Splunk Community!

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...