How to Stop logging certain type of logs in Splunk

[dnavia29]

Hello, I am facing problems of disk usage in Splunk and I've been asked to stop logging certain kinds of logs. I have read about Blacklists and Whitelists in order to ignore files but I am not able to manage that. All my logs are in /opt/config/logs/splunk and the log I'd like to stop logging has a type "itoken-app.log". I checked in the splunkforwarder to see if these logs as well were there but they don't appear in that route. Please help in giving me any idea to stop the itoken logs to be in my logs in splunk.

Thanks

[codebuilder · Accepted solution]

You can simply add this entry to your monitor stanza in /opt/splunkforwarder/etc/system/local/inputs.conf

e.g.

[monitor:///opt/config/logs/splunk]
blacklist = itoken-app.log

You will need to restart the UF for the change to take effect.

[dnavia29]

I tried it putting the complete name in the blacklist like you suggested and then restart the UF but it didn't work.. Should I use regex instead? since all the logs have the format at the end "app.log"

[codebuilder]

Yes, you may need to wildcard the log file name for blacklist'ing...

Try this:
blacklist = .-app.log$

[dnavia29]

No no, I meant the rest of the logs... for example, "security-app.log, transfers-superapp-cl-app.log, home-mobile-latam-app.log, cards-app.log, itoken-cl-app.log" and from that list I'd like to exclude the "itoken-cl-app.log", so should I use wildcard like "blacklist = itoken

[dnavia29]

(* )itoken(*)

[dnavia29]

It worked putting (|) with the name of the two logs that I needed to block, thanks for your help @codebuilder

[codebuilder]

Good deal, glad it's working for you and happy to help!

[dnavia29]

Thank you, I have another question regarding this issue.. If I want to exclude certain type of logs, the ones with the word "itoken" in it, but not all the logs that are in a file?.. Example:

I have files called "admin-app.log" and "admin-api.log", inside this files there are logs of "itoken", I want to exclude only the logs that contains "itoken" but not the other logs inside those files, any idea about how can I approach this? thanks

[codebuilder]

Ah, ok...just blacklist (backslash).itoken.$ then and you should be good.

Again, weird quirk with replies here but replace (backslash) the actual backslash character.

[codebuilder]

With the caveat that, as @richgalloway mentioned, if you are using a deployment server, this is not necessarily ideal. If you are not using one, this should get you by.

[codebuilder]

Ugh, Splunk forum page keeps stripping out slashes in my replies.
The syntax should be blacklist = (backslash).-app.log$'

Replace (backslash with "\")

[dnavia29]

I think I put it in the wrong way, what I meant was I have other logs with this format "app.log" for example "payments-app.log, transfers-app.log, security-app.log, transfers-cl-app.log, itoken-cl-app.log"... From all the logs with the same format, I only need to exclude "itoken-cl-app.log" so I cannot use wildcard at the end with the format because is going to exclude all the logs.. so Should I use wildcard like "blacklist = itoken

[dnavia29]

No I am not using a deployment server, I am trying this one in QA first

[richgalloway]

We shouldn't make changes to etc/system/local on a UF. It prevents the deployment server from overriding those changes later.

[codebuilder]

@richgalloway that is a very valid point. I was just trying to provide a quick example, but you are definitely correct.

[dnavia29]

Yes that's right, the inputs.conf I found it "/opt/splunkforwarder/etc/search/local" so that way prevents what you just said... Thank you, I think that might work.. I need to wait now to see if it's not logging anymore

[richgalloway]

Blacklists are the usual way to do that. Why can you not use them?

[dnavia29]

Because I see in the documentation and Saw this "Add the following line to your monitor stanza in the /local/inputs.conf file for the app context that you defined the input in.
blacklist = "

I am not sure if this it has to be done in the inputs.conf of the SplunkUniversalForwarder, and try to to a regular expression that excludes this one "itoken-cl-app.log" for example.

[richgalloway]

Yes, blacklist goes in the inputs.conf file on the UF. I don't understand how that prevents you from specifying a blacklist.