All Apps and Add-ons

How to monitor wildcard files recursively

signumpl
Engager

I want to index all *.log files recursively from /var/log
I followed this instruction https://docs.splunk.com/Documentation/Splunk/8.0.2/Data/Specifyinputpathswithwildcards
My inputs.conf looks like this:
[monitor:///var/log/]
whitelist=\.log$
recursive=true
disabled = false
index = rpi_logs
sourcetype = linux_logs

It seems to be indexing only /var/log/daemon.log and var/log/auth.log
But I also have log files in /var/log/mysql and /var/log/nginx directories and those are omitted. What am I doing wrong?

0 Karma

woodcock
Esteemed Legend

Like this:

[monitor:///var/log/.../*\.log$]
recursive=true
disabled = false 
index = rpi_logs 
sourcetype = linux_logs
0 Karma

ivanreis
Builder

please run the command ./splunk list inputstatus to troublehsoot the issue, this command will provide you an output like this:

this output means the input configuration is not working.

/var/log/installer/status
parent = /var/log
type = File did not match whitelist '(.log|log$|messages|secure|auth|mesg$|cron$|acpid$|.out)'.

this output means the file is being properly monitored by splunk
/var/log/installer/syslog
file position = 465885
file size = 465885
parent = /var/log
percent = 100.00
type = finished reading

It is possible the splunk is not able to read the log files from /var/log/mysql and /var/log/nginx, so I would try something like this:

[monitor:///var/log/.../]
whitelist=.log$
recursive=true
disabled = false
index = rpi_logs
sourcetype = linux_logs

Note concerning wildcards and monitor:
* You can use wildcards to specify your input path for monitored inputs. Use
"..." for recursive directory matching and "" for wildcard matching in a
single directory segment.
* "..." recurses through directories. This means that /foo/.../bar matches
foo/1/bar, foo/1/2/bar, etc.
* You can use multiple "..." specifications in a single input path. For
example: /foo/.../bar/...
* The asterisk (
) matches anything in a single path segment; unlike "...", it
does not recurse. For example, /foo//bar matches the files
/foo/1/bar, /foo/2/bar, etc. However, it does not match
/foo/bar or /foo/1/2/bar.
A second example: /foo/m*r/bar matches /foo/mr/bar, /foo/mir/bar,
/foo/moor/bar, etc. It does not match /foo/mi/or/bar.
* You can combine "
" and "..." as needed: foo/.../bar/* matches any file in
the bar directory within the specified path.

0 Karma

akshatj2
Path Finder

may be below inputs can help. let me know if this helps

[monitor:///var/log/.../*.log]
disabled = false
index = rpi_logs 
sourcetype = linux_logs
0 Karma
Get Updates on the Splunk Community!

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...