topic Re: File monitoring With an interval in Getting Data In

File monitoring With an interval

ninisimonishvil — Tue, 29 Sep 2020 19:35:30 GMT

Hello everyone.

I have an issue regarding monitoring files in a directory.

The thing is that in order events to be broken correctly the file must be first populated, so splunk must wait until it starts indexing that file. I created a stanza :

[monitor://C:\Program Files\mypath*]
disabled = false
index = default
sourcetype = mysourcetype
crcSalt = <SOURCE>
time_before_close = 900
multiline_event_extra_waittime = true

I set time_before_close to 15 minutes and multiline_event_extra_waittime to true, however, after trying this out, I see that splunk still does not wait for those 15 minutes to index the populated file and therefore does event breaking in a wrong way.

any suggestions?

Re: File monitoring With an interval

ansif — Fri, 08 Jun 2018 12:42:37 GMT

Restarted splunk service once updated inputs.conf file?

Re: File monitoring With an interval

jchivian — Tue, 29 Sep 2020 19:52:10 GMT

If the files are opened and written to continuously (like /var/log/messages) then you don't need the time_before_close or multiline_event_before_close, just correctly define the LINE_BREAKER and be done with it.

If the files are opened empty, slowly populated, and then closed never to be touched again, then you could modify the process such that they are created and populated with a temporary name, and then when closed are moved, copied, or renamed to something matching the monitoring criteria.

It's the age old problem when you disassociate the processes that create and require the results.