I've been messing about with this for a while now and I can't seem to figure out the rhyme or reason behind how wildcards work in the Splunk inputs.conf file. I'm trying to pull in logs from PingFederate... logs are in this directory:
Logs would look like:
server.log server.log.1 server.log.2 splunk-audit.log splunk-audit.2016-01-19.log splunk-audit.2016-01-20.log
I want to process the server.log file as well as the rollovers but none of my wildcards work. In my mind this should work... but it doesn't pull any files at all:
[monitor://E:\PingFederate-Engine\log\server*.log*] index = pingfederate_server [monitor://E:\PingFederate-Engine\log\splunk_audit*.log] index = pingfederate_splunk_audit
Any idea what the trick is behind these wildcards?
You can have multiple monitor stanza (for same directory) if you're monitoring specific files. so this should work just fine.
[monitor://E:\PingFederate-Engine\log\server*.log] index = pingfederate_server [monitor://E:\PingFederate-Engine\log\splunk_audit.log] index = pingfederate_splunk_audit
Ensure that you restart your splunk instance where you configured this. Also, after you restart, run following command from the command line to see if the monitoring is able to pick up required files or not. (after going to Splunk Installation directory)
splunk.exe list monitor
BTW, why do you want to read the rollover files?? If the logs are written only of regular files then Splunk will not read the rollover files, even when you specifically monitor them.
Also, are the files being written currently OR they are old files and you just want to ingest them?
Hey there, I think I'm getting somewhere based on running the list monitor command... why is it listing it as a monitored directory and not a file?:
Monitored Directories: $SPLUNK_HOME\var\log\splunk C:\SplunkUniversalForwarder\var\log\splunk\audit.log C:\SplunkUniversalForwarder\var\log\splunk\btool.log C:\SplunkUniversalForwarder\var\log\splunk\conf.log C:\SplunkUniversalForwarder\var\log\splunk\first_install.log C:\SplunkUniversalForwarder\var\log\splunk\license_audit.log C:\SplunkUniversalForwarder\var\log\splunk\license_usage.log C:\SplunkUniversalForwarder\var\log\splunk\mongod.log C:\SplunkUniversalForwarder\var\log\splunk\remote_searches.log C:\SplunkUniversalForwarder\var\log\splunk\scheduler.log C:\SplunkUniversalForwarder\var\log\splunk\searchhistory.log C:\SplunkUniversalForwarder\var\log\splunk\splunkd-utility.log C:\SplunkUniversalForwarder\var\log\splunk\splunkd_access.log C:\SplunkUniversalForwarder\var\log\splunk\splunkd_ui_access.log $SPLUNK_HOME\var\log\splunk\metrics.log C:\SplunkUniversalForwarder\var\log\splunk\metrics.log $SPLUNK_HOME\var\log\splunk\splunkd.log C:\SplunkUniversalForwarder\var\log\splunk\splunkd.log $SPLUNK_HOME\var\spool\splunk\...stash_new E:\PingFederate-Engine\log\splunk_audit*.log Monitored Files: $SPLUNK_HOME\etc\splunk.version
BTW, why do you want to read the
rollover files?? If the logs are
written only of regular files then
Splunk will not read the rollover
files, even when you specifically
monitor them. Also, are the files
being written currently OR they are
old files and you just want to ingest
I was worried that Splunk might miss a very small amount of logging data between the last time it forwarded log information and when the log files rolled over for the next day.
Splunk does take care of rolling over files (read the last unread content), so you should be good. What I would suggest is to read just the base file splunk_audit.log and change your monitoring stanza to remove wildcard. Redo the same steps (restart and list monitoring) and let's see if that helps.
I had read in another Splunk Answers thread that you can't have two monitors, so I tried just this:
[monitor://E:\PingFederate-Engine\log\splunk_audit*.log] index = pingfederate
And that still doesn't pick it up... I had this problem on a Syslog server where I had several different Syslog files for different applications I was monitoring but none of the wildcarding worked. I tried using whitelists as well to no avail... does the * not work when used in a monitor stanza for some reason?