All Apps and Add-ons
Highlighted

How to monitor wildcard files recursively

Engager

I want to index all *.log files recursively from /var/log
I followed this instruction https://docs.splunk.com/Documentation/Splunk/8.0.2/Data/Specifyinputpathswithwildcards
My inputs.conf looks like this:
[monitor:///var/log/]
whitelist=\.log$
recursive=true
disabled = false
index = rpi_logs
sourcetype = linux_logs

It seems to be indexing only /var/log/daemon.log and var/log/auth.log
But I also have log files in /var/log/mysql and /var/log/nginx directories and those are omitted. What am I doing wrong?

0 Karma
Highlighted

Re: How to monitor wildcard files recursively

Path Finder

may be below inputs can help. let me know if this helps

[monitor:///var/log/.../*.log]
disabled = false
index = rpi_logs 
sourcetype = linux_logs
0 Karma
Highlighted

Re: How to monitor wildcard files recursively

Builder

please run the command ./splunk list inputstatus to troublehsoot the issue, this command will provide you an output like this:

this output means the input configuration is not working.

/var/log/installer/status
parent = /var/log
type = File did not match whitelist '(.log|log$|messages|secure|auth|mesg$|cron$|acpid$|.out)'.

this output means the file is being properly monitored by splunk
/var/log/installer/syslog
file position = 465885
file size = 465885
parent = /var/log
percent = 100.00
type = finished reading

It is possible the splunk is not able to read the log files from /var/log/mysql and /var/log/nginx, so I would try something like this:

[monitor:///var/log/.../]
whitelist=.log$
recursive=true
disabled = false
index = rpilogs
sourcetype = linux
logs

Note concerning wildcards and monitor:
* You can use wildcards to specify your input path for monitored inputs. Use
"..." for recursive directory matching and "" for wildcard matching in a
single directory segment.
* "..." recurses through directories. This means that /foo/.../bar matches
foo/1/bar, foo/1/2/bar, etc.
* You can use multiple "..." specifications in a single input path. For
example: /foo/.../bar/...
* The asterisk (
) matches anything in a single path segment; unlike "...", it
does not recurse. For example, /foo//bar matches the files
/foo/1/bar, /foo/2/bar, etc. However, it does not match
/foo/bar or /foo/1/2/bar.
A second example: /foo/m
r/bar matches /foo/mr/bar, /foo/mir/bar,
/foo/moor/bar, etc. It does not match /foo/mi/or/bar.
* You can combine "" and "..." as needed: foo/.../bar/ matches any file in
the bar directory within the specified path.

0 Karma
Highlighted

Re: How to monitor wildcard files recursively

Esteemed Legend

Like this:

[monitor:///var/log/.../*\.log$]
recursive=true
disabled = false 
index = rpi_logs 
sourcetype = linux_logs
0 Karma