Getting Data In

What is the file and variable in "Splunk Add-on AWS" for S3, that limits the ingestion of files to 1 hour?

acceo_purch
New Member

Hi,

Please, Can some one let me know what is the file and variable in "Splunk Add-on AWS" for S3, that limits the ingestion of files to 1 hour? I didn't find in inputs.conf file any variable that limits the ingestion of files to 1 hour.

We need to index older files from S3 bucket but "Splunk Add-on AWS" only let index the last hour.

This is the inputs.conf file

[aws_s3://cloud-logs]
aws_account = abc
aws_s3_region = us-east-1
bucket_name = f-logs
character_set = auto
ct_blacklist = ^$
host_name = s3.us-east-1.amazonaws.com
index = cloud
initial_scan_datetime = 2022-01-14T15:59:18Z
max_items = 100000
max_retries = 3
polling_interval = 300
private_endpoint_enabled = 0
recursion_depth = -1
sourcetype = cloud:json
disabled = 0

Regards

Edgard Patino

 

 

 

Labels (2)
0 Karma

nyc_jason
Splunk Employee
Splunk Employee

Are you looking for log_start_date? See here in the example (which has it under setting up from the UI, though can should be able to do it directly when editing the .conf files too) https://docs.splunk.com/Documentation/AddOns/released/AWS/S3#Configure_a_Generic_S3_input_using_conf...

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...