Getting Data In

File monitoring With an interval

ninisimonishvil
Path Finder

Hello everyone.

I have an issue regarding monitoring files in a directory.

The thing is that in order events to be broken correctly the file must be first populated, so splunk must wait until it starts indexing that file. I created a stanza :

[monitor://C:\Program Files\mypath*]
disabled = false
index = default
sourcetype = mysourcetype
crcSalt = <SOURCE>
time_before_close = 900
multiline_event_extra_waittime = true

I set time_before_close to 15 minutes and multiline_event_extra_waittime to true, however, after trying this out, I see that splunk still does not wait for those 15 minutes to index the populated file and therefore does event breaking in a wrong way.

any suggestions?

0 Karma

ansif
Motivator

Restarted splunk service once updated inputs.conf file?

0 Karma

jchivian
Explorer

If the files are opened and written to continuously (like /var/log/messages) then you don't need the time_before_close or multiline_event_before_close, just correctly define the LINE_BREAKER and be done with it.

If the files are opened empty, slowly populated, and then closed never to be touched again, then you could modify the process such that they are created and populated with a temporary name, and then when closed are moved, copied, or renamed to something matching the monitoring criteria.

It's the age old problem when you disassociate the processes that create and require the results.

Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...