Getting Data In

How to monitor a folder to index each new text file every day?

paola92
Explorer

Hi,

I have a search head and I need to monitor a folder that has a text file in which every day there is a new file. I configured the Splunk forwarder on the host and configured Splunk for monitoring the folder, but I only receive one file and it never shows more.

What must I configure to receive the rest of files?

0 Karma

ddrillic
Ultra Champion

Based on the following the ? mark is not a valid syntax here Specify input paths with wildcards

alt text

You can also remove the recursive = true. We had bad experience with it ; -) at Splunk not matching files with wildcard in monitor path in inputs.conf

0 Karma

gokadroid
Motivator

If files are always going to be *.txt, can you monitor C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\*.txt rather than just the directory C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\

If file extension formats to be monitored are different, try with C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\* and see if it solves your issue.

0 Karma

paola92
Explorer

The configuration in the splunk web is the same in the space of File or Directory or can I put C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\?

0 Karma

paola92
Explorer

The files are XML for example a file has the name LogSwitchLight09-05-2016.txt and the initial line is: 001250 20160905

and other file is LogSwitchLight09-29-2016 and the initial line is: 001179 20160929

I have a inputs.conf in the path: C:\Program Files\SplunkUniversalForwarder\etc\apps\SplunkUniversalForwarder\local
and the text is:

[monitor://C:\Users\paola.sarmiento\Documents\CFA-Splunk\Log\]
disabled = false
recursive = true
index=main
0 Karma

maciep
Champion

have you tried searching the internal logs from that server? Maybe restart the forwarder and then review the logs to see if splunk throws any warning/errors about those files that are getting indexed.

0 Karma

maciep
Champion

What kind of file is it? Does it have a header line maybe? Splunk identifies a file by the first xx bytes, so if all of the files have a common header, splunk may think it's just various copies of the same files, so it won't re-ingest it. There are ways around that.

Also, posting the monitor stanza from your inputs.conf could be helpful too.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...