topic Re: How to monitor wildcard files recursively in All Apps and Add-ons

How to monitor wildcard files recursively

signumpl — Tue, 10 Mar 2020 22:01:34 GMT

I want to index all *.log files recursively from /var/log
I followed this instruction https://docs.splunk.com/Documentation/Splunk/8.0.2/Data/Specifyinputpathswithwildcards
My inputs.conf looks like this:
[monitor:///var/log/] whitelist=\.log$ recursive=true disabled = false index = rpi_logs sourcetype = linux_logs

It seems to be indexing only /var/log/daemon.log and var/log/auth.log
But I also have log files in /var/log/mysql and /var/log/nginx directories and those are omitted. What am I doing wrong?

Re: How to monitor wildcard files recursively

akshatj2 — Tue, 10 Mar 2020 23:04:02 GMT

may be below inputs can help. let me know if this helps

[monitor:///var/log/.../*.log]
disabled = false
index = rpi_logs 
sourcetype = linux_logs

Re: How to monitor wildcard files recursively

ivanreis — Wed, 30 Sep 2020 04:36:26 GMT

please run the command ./splunk list inputstatus to troublehsoot the issue, this command will provide you an output like this:

this output means the input configuration is not working.

this output means the file is being properly monitored by splunk
/var/log/installer/syslog
file position = 465885
file size = 465885
parent = /var/log
percent = 100.00
type = finished reading

It is possible the splunk is not able to read the log files from /var/log/mysql and /var/log/nginx, so I would try something like this:

[monitor:///var/log/.../]
whitelist=.log$
recursive=true
disabled = false
index = rpi_logs
sourcetype = linux_logs

Note concerning wildcards and monitor:
* You can use wildcards to specify your input path for monitored inputs. Use
"..." for recursive directory matching and "" for wildcard matching in a
single directory segment.
* "..." recurses through directories. This means that /foo/.../bar matches
foo/1/bar, foo/1/2/bar, etc.
* You can use multiple "..." specifications in a single input path. For
example: /foo/.../bar/...
* The asterisk () matches anything in a single path segment; unlike "...", it
does not recurse. For example, /foo//bar matches the files
/foo/1/bar, /foo/2/bar, etc. However, it does not match
/foo/bar or /foo/1/2/bar.
A second example: /foo/m*r/bar matches /foo/mr/bar, /foo/mir/bar,
/foo/moor/bar, etc. It does not match /foo/mi/or/bar.
* You can combine "" and "..." as needed: foo/.../bar/* matches any file in
the bar directory within the specified path.

Re: How to monitor wildcard files recursively

woodcock — Wed, 11 Mar 2020 00:31:55 GMT

Like this:

[monitor:///var/log/.../*\.log$]
recursive=true
disabled = false 
index = rpi_logs 
sourcetype = linux_logs