Getting Data In

How to use blacklist in inputs.conf?

knalla
Path Finder

Hi,

How do you edit inputs.conf to blacklist some hosts from indexing and index those hosts to different index?

list of the servers:

/opt/logs/
server1
server2
server3
server4
server5
server6

inputs.conf

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = abc
blacklist=(server4|server5)
sourcetype = abc
blacklist = .gz$

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = xyz
whitelsit=(server4|server5)
sourcetype = abc
blacklist = .gz$

0 Karma

sudosplunk
Motivator

If a file matches the regexes in both the blacklist and whitelist settings, the file is NOT monitored. Blacklists take precedence over whitelists.

Try this combination in inputs.conf and see of it works,

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = abc
whitelist=(server1|server2|server3|server6)
sourcetype = abc
blacklist = .gz$

[monitor:///opt/logs/*/*log]
disabled = 0
host_segment = 4
index = xyz
whitelsit=(server4|server5)
sourcetype = abc
blacklist = .gz$

Also, based on your monitor statement, I don't think host_segment=4 will pick up directory name as hostname. Try host_segment=3.

For example, if you set host_segment=3 and the monitor path is /opt/logs/host01/some.log, Splunk software sets the host as "host01" because that is the third segment.

knalla
Path Finder

Thanks for the response, I have multiple hosts to white list around 200 and black list around 10.

can I use 2 blacklists in a stanza, one for the hosts and one for .gz$?

0 Karma

sudosplunk
Motivator

Since blacklist supports regex, you can define regex to capture all 200 OR 10 hosts. Let me know how your hostname(s) looks like and I will try to provide a regex.

To my knowledge, you should be able to use 2 blacklists but be sure to number them, blacklist1, blacklist2, blacklist3 so on. More details here.

Alternatively, this should also work, blacklist = server1|\.gz$

0 Karma

pruthvikrishnap
Contributor

Hi,
You can do this, please find the docs below.
https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Filter_data_by_ta...
you will have to blacklist them and assign to a different group name and mention that in outputs.conf

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...