Getting Data In

How to monitor a folder to index each new text file every day?

paola92
Explorer

Hi,

I have a search head and I need to monitor a folder that has a text file in which every day there is a new file. I configured the Splunk forwarder on the host and configured Splunk for monitoring the folder, but I only receive one file and it never shows more.

What must I configure to receive the rest of files?

0 Karma

ddrillic
Ultra Champion

Based on the following the ? mark is not a valid syntax here Specify input paths with wildcards

alt text

You can also remove the recursive = true. We had bad experience with it ; -) at Splunk not matching files with wildcard in monitor path in inputs.conf

0 Karma

gokadroid
Motivator

If files are always going to be *.txt, can you monitor C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\*.txt rather than just the directory C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\

If file extension formats to be monitored are different, try with C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\* and see if it solves your issue.

0 Karma

paola92
Explorer

The configuration in the splunk web is the same in the space of File or Directory or can I put C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\?

0 Karma

paola92
Explorer

The files are XML for example a file has the name LogSwitchLight09-05-2016.txt and the initial line is: 001250 20160905

and other file is LogSwitchLight09-29-2016 and the initial line is: 001179 20160929

I have a inputs.conf in the path: C:\Program Files\SplunkUniversalForwarder\etc\apps\SplunkUniversalForwarder\local
and the text is:

[monitor://C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\]
disabled = false
recursive = true
index=main
0 Karma

maciep
Champion

have you tried searching the internal logs from that server? Maybe restart the forwarder and then review the logs to see if splunk throws any warning/errors about those files that are getting indexed.

0 Karma

maciep
Champion

What kind of file is it? Does it have a header line maybe? Splunk identifies a file by the first xx bytes, so if all of the files have a common header, splunk may think it's just various copies of the same files, so it won't re-ingest it. There are ways around that.

Also, posting the monitor stanza from your inputs.conf could be helpful too.

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!