Getting Data In
Highlighted

How to monitor a folder to index each new text file every day?

Explorer

Hi,

I have a search head and I need to monitor a folder that has a text file in which every day there is a new file. I configured the Splunk forwarder on the host and configured Splunk for monitoring the folder, but I only receive one file and it never shows more.

What must I configure to receive the rest of files?

0 Karma
Highlighted

Re: How to monitor a folder to index each new text file every day?

Champion

What kind of file is it? Does it have a header line maybe? Splunk identifies a file by the first xx bytes, so if all of the files have a common header, splunk may think it's just various copies of the same files, so it won't re-ingest it. There are ways around that.

Also, posting the monitor stanza from your inputs.conf could be helpful too.

0 Karma
Highlighted

Re: How to monitor a folder to index each new text file every day?

Explorer

The files are XML for example a file has the name LogSwitchLight09-05-2016.txt and the initial line is: 001250 20160905

and other file is LogSwitchLight09-29-2016 and the initial line is: 001179 20160929

I have a inputs.conf in the path: C:\Program Files\SplunkUniversalForwarder\etc\apps\SplunkUniversalForwarder\local
and the text is:

[monitor://C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\]
disabled = false
recursive = true
index=main
0 Karma
Highlighted

Re: How to monitor a folder to index each new text file every day?

Champion

have you tried searching the internal logs from that server? Maybe restart the forwarder and then review the logs to see if splunk throws any warning/errors about those files that are getting indexed.

0 Karma
Highlighted

Re: How to monitor a folder to index each new text file every day?

Motivator

If files are always going to be *.txt, can you monitor C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\*.txt rather than just the directory C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\

If file extension formats to be monitored are different, try with C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\* and see if it solves your issue.

0 Karma
Highlighted

Re: How to monitor a folder to index each new text file every day?

Explorer

The configuration in the splunk web is the same in the space of File or Directory or can I put C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\?

0 Karma
Highlighted

Re: How to monitor a folder to index each new text file every day?

Ultra Champion

Based on the following the ? mark is not a valid syntax here Specify input paths with wildcards

alt text

You can also remove the recursive = true. We had bad experience with it ; -) at Splunk not matching files with wildcard in monitor path in inputs.conf

0 Karma