https://community.splunk.com/t5/Getting-Data-In/Why-is-time-before-close-attribute-causing-a-delay-in-indexing/m-p/114071#M23846 <P>I know this question is older but I will try to answer it since I hit on it an think maybe the answer is interesting for the 765 guys that have seen this question till now and the ones that will see this question.</P> <P>By default Splunk will only open up to 100 file descriptors for reading logs (limits.conf : max_fd).<BR /> So if you define time_before_close and monitor more than the limit of file descriptors splunk first has to wait until a file discriptor becomes free before it can start reading the next file.</P> <P>So if you have many monitored files splunk will imediatly start reading the first 100 files, wait time_before_close (in your case 5mins) and after that starts reading the next 100 files ...</P> Tue, 29 Sep 2020 07:58:09 GMT doitslu 2020-09-29T07:58:09Z https://community.splunk.com/t5/Getting-Data-In/Why-is-time-before-close-attribute-causing-a-delay-in-indexing/m-p/114070#M23845 <P>I had set the value of time_before_close attribute to 300 (5 mins) in one of my monitor stanzas. What I observed is that splunk indexes the logs for the first few seconds and then stops indexing for the next 5 mins. After 5 mins next set of logs are indexed by splunk and then continues to wait for 5 mins.</P> <P>I was expecting that time_before_close attribute would just keep the file handler open for the specified seconds after the last line was written in to the file but with continues indexing.</P> <P>Any thoughts on why I see this behavior ? Or can someone direct me to somewhere I can find the answer ?</P> Mon, 28 Sep 2020 19:20:01 GMT https://community.splunk.com/t5/Getting-Data-In/Why-is-time-before-close-attribute-causing-a-delay-in-indexing/m-p/114070#M23845 nibinabr 2020-09-28T19:20:01Z https://community.splunk.com/t5/Getting-Data-In/Why-is-time-before-close-attribute-causing-a-delay-in-indexing/m-p/114071#M23846 <P>I know this question is older but I will try to answer it since I hit on it an think maybe the answer is interesting for the 765 guys that have seen this question till now and the ones that will see this question.</P> <P>By default Splunk will only open up to 100 file descriptors for reading logs (limits.conf : max_fd).<BR /> So if you define time_before_close and monitor more than the limit of file descriptors splunk first has to wait until a file discriptor becomes free before it can start reading the next file.</P> <P>So if you have many monitored files splunk will imediatly start reading the first 100 files, wait time_before_close (in your case 5mins) and after that starts reading the next 100 files ...</P> Tue, 29 Sep 2020 07:58:09 GMT https://community.splunk.com/t5/Getting-Data-In/Why-is-time-before-close-attribute-causing-a-delay-in-indexing/m-p/114071#M23846 doitslu 2020-09-29T07:58:09Z