topic Re: how to setup forwarder to ingest logs from a specific date / last day's log and carry on monitoring the future files as they are created? in Getting Data In

how to setup forwarder to ingest logs from a specific date / last day's log and carry on monitoring the future files as they are created?

shivarpith — Mon, 28 Mar 2016 15:44:52 GMT

i am dealing with a imilar issue, i am trying to ingest webserver logs and the historical log data in webserver is huge and brought splunk down when i tried to ingest. i want splunk UF to start from the latest or the day before's log file and ingest the new log files that are created in future

Example:
exclude logs till march 27th.
ingest march 28th.log
march 29th.log
march 30th.log
march 31th.log
april 1st.log

.
.
.
.
.
and all log files post march 28th ( normal splunk UF behaviour)

can i use current_only=1 setting or any other suggested recommendation.

Hope i was clear enough. Thanks in advance

Re: how to setup forwarder to ingest logs from a specific date / last day's log and carry on monitoring the future files as they are created?

martin_mueller — Mon, 28 Mar 2016 16:45:59 GMT

You can set ignoreOlderThan=3d in inputs.conf to do just that, ignore files older than three days or whatever time span you like.

Re: how to setup forwarder to ingest logs from a specific date / last day's log and carry on monitoring the future files as they are created?

ddrillic — Mon, 28 Mar 2016 17:46:58 GMT

Just came across the ignoreOlderThan topic at https://answers.splunk.com/topics/ignoreolderthan.html.