Syslog File Monitor - Issues with multiple monitor paths

[fairje]

For some reason the inputs.conf is not liking how I am giving it two monitor paths with wildcards in the same set of subdirectories and it is causing issues.

Inputs:

    [monitor:///opt/log/192.168.1.(37|38|39|40)*/Juniper.log]
    disabled = 0
    host_segment = 3
    index = ssl_vpn
    sourcetype = juniper_sa_log
    [monitor:///opt/log/LoadBalancer0(2|3|4)*/*.log]
    disabled = 0
    host_segment = 3
    index = f5
    sourcetype = f5:bigip:syslog

Note I have tried multiple variations of either using whitelists, not using whitelists, some wildcards mixed with options or just straight wildcards and yet I get some conflict in the splunk list monitor

Monitored Directories:
        /opt/log/LoadBalancer0(2|3|4)*/*.log
                /opt/log/192.168.1.37
                /opt/log/192.168.1.37/Juniper.log
                /opt/log/192.168.1.38
                /opt/log/192.168.1.38/Juniper.log
                /opt/log/192.168.1.39
                /opt/log/192.168.1.39/Juniper.log
                /opt/log/192.168.1.40
                /opt/log/192.168.1.40/Juniper.log
                /opt/log/LoadBalancer02
                /opt/log/LoadBalancer02/bigpipe.log
                /opt/log/LoadBalancer02/crond.log
                /opt/log/LoadBalancer02/gtmd.log
                /opt/log/LoadBalancer02/httpd(pam_audit).log
                /opt/log/LoadBalancer02/httpd.log
                /opt/log/LoadBalancer02/logger.log
                /opt/log/LoadBalancer02/mcpd.log
                /opt/log/LoadBalancer02/syslog-ng.log
                /opt/log/LoadBalancer03
                /opt/log/LoadBalancer03/bigpipe.log
                /opt/log/LoadBalancer03/crond.log
                /opt/log/LoadBalancer03/gtmd.log
                /opt/log/LoadBalancer03/httpd.log
                /opt/log/LoadBalancer03/logger.log
                /opt/log/LoadBalancer03/mcpd.log
                /opt/log/LoadBalancer03/mprov.log
                /opt/log/LoadBalancer03/ntpd.log
                /opt/log/LoadBalancer03/restorecond.log
                /opt/log/LoadBalancer03/snmpd.log
                /opt/log/LoadBalancer03/syslog-ng.log
                /opt/log/LoadBalancer03/usermod.log
                /opt/log/LoadBalancer04
                /opt/log/LoadBalancer04/bigpipe.log
                /opt/log/LoadBalancer04/crond.log
                /opt/log/LoadBalancer04/f5mku.log
                /opt/log/LoadBalancer04/gtmd.log
                /opt/log/LoadBalancer04/httpd(pam_audit).log
                /opt/log/LoadBalancer04/httpd.log
                /opt/log/LoadBalancer04/logger.log
                /opt/log/LoadBalancer04/mcpd.log
                /opt/log/LoadBalancer04/snmpd.log
                /opt/log/LoadBalancer04/syslog-ng.log
Monitored Files:
        $SPLUNK_HOME/etc/splunk.version
        /Library/Logs
        /opt/log/192.168.1.(37|38|39|40)*/Juniper.log
        /root/.bash_history
        /var/adm

So you can see the one monitor path is taking precidence over the other one and blindly applying both to it. But it is taking the filtering accurately across the two because there are other folders under /opt/log which are not showing up. Any ideas?

[tskinnerivsec]

I'm pretty sure this is because a file monitor is a regex, and your override is happening because you are using * as a wild card at the end of the same directory level. The 1st file monitor that matches will take precedence. Try defining your file monitors like:

[monitor:///opt/log/192.168.1.(37|38|39|40)/Juniper.log] and
[monitor:///opt/log/LoadBalancer0(2|3|4)/*.log]

[fairje]

According to: http://docs.splunk.com/Documentation/Splunk/6.2.5/Data/Specifyinputpathswithwildcards

It needs to have a wildcard of some kind in that section of the stanza in order to activate the usage of regex. I have tried this (before I went and actually read the linked document) and it didn't work, unfortunately.

Why, oh why, can't they just use full on regex in this field instead of making it so complicated -_-

[tskinnerivsec]

looks like in this case, you'll just have to use a wildcard earlier in your statement. Will it still work if you use:

[monitor:///opt/log/192.168.*.(37|38|39|40)/Juniper.log]

I have consistently run into issues with tailing wildcards when configuring multiple file monitors to act recursively from a higher point in the directory tree.