topic How to use blacklist in inputs.conf? in Getting Data In

How to use blacklist in inputs.conf?

knalla — Tue, 29 Sep 2020 21:02:49 GMT

Hi,

How do you edit inputs.conf to blacklist some hosts from indexing and index those hosts to different index?

list of the servers:

/opt/logs/
server1
server2
server3
server4
server5
server6

inputs.conf

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = abc
blacklist=(server4|server5)
sourcetype = abc
blacklist = .gz$

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = xyz
whitelsit=(server4|server5)
sourcetype = abc
blacklist = .gz$

Re: How to use blacklist in inputs.conf?

pruthvikrishnap — Tue, 28 Aug 2018 21:14:27 GMT

Hi,
You can do this, please find the docs below.
https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Filter_data_by_target_index
you will have to blacklist them and assign to a different group name and mention that in outputs.conf

Re: How to use blacklist in inputs.conf?

Shan — Wed, 29 Aug 2018 11:40:27 GMT

@knalla,

Please refer to previous answers. Let me know if it helps you or not.

https://answers.splunk.com/answers/141957/blacklist-file-form-inputs-conf.html

https://answers.splunk.com/answers/134091/inputs-conf-whitelist-blacklist-question.html

https://answers.splunk.com/answers/542588/how-to-edit-inputsconf-to-blacklist-an-eventcode.html

http://docs.splunk.com/Documentation/SplunkCloud/latest/Data/Whitelistorblacklistspecificincomingdata

Re: How to use blacklist in inputs.conf?

sudosplunk — Wed, 29 Aug 2018 13:39:41 GMT

If a file matches the regexes in both the blacklist and whitelist settings, the file is NOT monitored. Blacklists take precedence over whitelists.

Try this combination in inputs.conf and see of it works,

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = abc
whitelist=(server1|server2|server3|server6)
sourcetype = abc
blacklist = .gz$

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = xyz
whitelsit=(server4|server5)
sourcetype = abc
blacklist = .gz$

Also, based on your monitor statement, I don't think host_segment=4 will pick up directory name as hostname. Try host_segment=3.

For example, if you set host_segment=3 and the monitor path is /opt/logs/host01/some.log, Splunk software sets the host as "host01" because that is the third segment.

Re: How to use blacklist in inputs.conf?

knalla — Thu, 30 Aug 2018 16:03:41 GMT

Thanks for the response, I have multiple hosts to white list around 200 and black list around 10.

can I use 2 blacklists in a stanza, one for the hosts and one for .gz$?

Re: How to use blacklist in inputs.conf?

sudosplunk — Thu, 30 Aug 2018 17:56:07 GMT

Since blacklist supports regex, you can define regex to capture all 200 OR 10 hosts. Let me know how your hostname(s) looks like and I will try to provide a regex.

To my knowledge, you should be able to use 2 blacklists but be sure to number them, blacklist1, blacklist2, blacklist3 so on. More details here.

Alternatively, this should also work, blacklist = server1|\.gz$