i am dealing with a imilar issue, i am trying to ingest webserver logs and the historical log data in webserver is huge and brought splunk down when i tried to ingest. i want splunk UF to start from the latest or the day before's log file and ingest the new log files that are created in future
Example:
exclude logs till march 27th.
ingest march 28th.log
march 29th.log
march 30th.log
march 31th.log
april 1st.log
.
.
.
.
.
and all log files post march 28th ( normal splunk UF behaviour)
can i use current_only=1 setting or any other suggested recommendation.
Hope i was clear enough. Thanks in advance
You can set ignoreOlderThan=3d
in inputs.conf to do just that, ignore files older than three days or whatever time span you like.
Just came across the ignoreOlderThan topic at https://answers.splunk.com/topics/ignoreolderthan.html.