All Apps and Add-ons

How to whitelist a pattern of a file in multiple folders in AWS splunk add on

New Member

I have s3 bucket containing folders of format i-0XXXXXXXXX .
each of these folders has a log file of this pattern XXXqueriesXXX.gz.
My key prefix (The path to the i-0XXXXXX folders) looks something like this resources/logs/e-muretrsd/.

Basically, I am looking to pull logs from locations satisfying this pattern resources/logs/e-mustt/i-XXXXXXX/XXXXXXqueriesXXXXX.gz

How can I achieve this in the splunk aws addon

0 Karma

Champion

Hi,

Please refer below link:
https://docs.splunk.com/Documentation/AddOns/released/AWS/S3
https://docs.splunk.com/Documentation/Splunk/7.0.2/Admin/Inputsconf

Also in inputs.conf use whitelist parameter using regex something like this:

[input_stanza]
..
whitelist = resources\/logs\/e-mustt\/i\-.+?/.+queries.+\.gz$
0 Karma

New Member

so if I use i-.+? we get the all the folders starting with i-XXXX in the directory?

0 Karma

Engager

I know this old post but were able to get this solved? I'm having the same issue but not finding much in the way of documentation S3 key prefix. 

0 Karma

Explorer

Likewise suffering a lack of documentation on the use of the AWS configuration settings.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!