How to use blacklist in inputs.conf?

knalla · ‎08-28-2018

Hi,

How do you edit inputs.conf to blacklist some hosts from indexing and index those hosts to different index?

list of the servers:

/opt/logs/
server1
server2
server3
server4
server5
server6

inputs.conf

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = abc
blacklist=(server4|server5)
sourcetype = abc
blacklist = .gz$

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = xyz
whitelsit=(server4|server5)
sourcetype = abc
blacklist = .gz$

sudosplunk · ‎08-29-2018

If a file matches the regexes in both the blacklist and whitelist settings, the file is NOT monitored. Blacklists take precedence over whitelists.

Try this combination in inputs.conf and see of it works,

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = abc
whitelist=(server1|server2|server3|server6)
sourcetype = abc
blacklist = .gz$

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = xyz
whitelsit=(server4|server5)
sourcetype = abc
blacklist = .gz$

Also, based on your monitor statement, I don't think host_segment=4 will pick up directory name as hostname. Try host_segment=3.

For example, if you set host_segment=3 and the monitor path is /opt/logs/host01/some.log, Splunk software sets the host as "host01" because that is the third segment.

knalla · ‎08-30-2018

Thanks for the response, I have multiple hosts to white list around 200 and black list around 10.

can I use 2 blacklists in a stanza, one for the hosts and one for .gz$?

sudosplunk · ‎08-30-2018

Since blacklist supports regex, you can define regex to capture all 200 OR 10 hosts. Let me know how your hostname(s) looks like and I will try to provide a regex.

To my knowledge, you should be able to use 2 blacklists but be sure to number them, blacklist1, blacklist2, blacklist3 so on. More details here.

Alternatively, this should also work, blacklist = server1|\.gz$