Getting Data In

How to Stop logging certain type of logs in Splunk

dnavia29
New Member

Hello, I am facing problems of disk usage in Splunk and I've been asked to stop logging certain kinds of logs. I have read about Blacklists and Whitelists in order to ignore files but I am not able to manage that. All my logs are in /opt/config/logs/splunk and the log I'd like to stop logging has a type "itoken-app.log". I checked in the splunkforwarder to see if these logs as well were there but they don't appear in that route. Please help in giving me any idea to stop the itoken logs to be in my logs in splunk.

Thanks

0 Karma
1 Solution

codebuilder
Influencer

You can simply add this entry to your monitor stanza in /opt/splunkforwarder/etc/system/local/inputs.conf

e.g.

[monitor:///opt/config/logs/splunk]
blacklist = itoken-app.log

You will need to restart the UF for the change to take effect.

----
An upvote would be appreciated and Accept Solution if it helps!

View solution in original post

0 Karma

codebuilder
Influencer

You can simply add this entry to your monitor stanza in /opt/splunkforwarder/etc/system/local/inputs.conf

e.g.

[monitor:///opt/config/logs/splunk]
blacklist = itoken-app.log

You will need to restart the UF for the change to take effect.

----
An upvote would be appreciated and Accept Solution if it helps!

View solution in original post

0 Karma

dnavia29
New Member

I tried it putting the complete name in the blacklist like you suggested and then restart the UF but it didn't work.. Should I use regex instead? since all the logs have the format at the end "app.log"

0 Karma

codebuilder
Influencer

Yes, you may need to wildcard the log file name for blacklist'ing...

Try this:
blacklist = .-app.log$

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

dnavia29
New Member

No no, I meant the rest of the logs... for example, "security-app.log, transfers-superapp-cl-app.log, home-mobile-latam-app.log, cards-app.log, itoken-cl-app.log" and from that list I'd like to exclude the "itoken-cl-app.log", so should I use wildcard like "blacklist = itoken

0 Karma

dnavia29
New Member

(* )itoken(*)

0 Karma

dnavia29
New Member

It worked putting (|) with the name of the two logs that I needed to block, thanks for your help @codebuilder

0 Karma

codebuilder
Influencer

Good deal, glad it's working for you and happy to help!

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

dnavia29
New Member

Thank you, I have another question regarding this issue.. If I want to exclude certain type of logs, the ones with the word "itoken" in it, but not all the logs that are in a file?.. Example:

I have files called "admin-app.log" and "admin-api.log", inside this files there are logs of "itoken", I want to exclude only the logs that contains "itoken" but not the other logs inside those files, any idea about how can I approach this? thanks

0 Karma

codebuilder
Influencer

Ah, ok...just blacklist (backslash).itoken.$ then and you should be good.

Again, weird quirk with replies here but replace (backslash) the actual backslash character.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

codebuilder
Influencer

With the caveat that, as @richgalloway mentioned, if you are using a deployment server, this is not necessarily ideal. If you are not using one, this should get you by.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

codebuilder
Influencer

Ugh, Splunk forum page keeps stripping out slashes in my replies.
The syntax should be blacklist = (backslash).-app.log$'

Replace (backslash with "\")

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

dnavia29
New Member

I think I put it in the wrong way, what I meant was I have other logs with this format "app.log" for example "payments-app.log, transfers-app.log, security-app.log, transfers-cl-app.log, itoken-cl-app.log"... From all the logs with the same format, I only need to exclude "itoken-cl-app.log" so I cannot use wildcard at the end with the format because is going to exclude all the logs.. so Should I use wildcard like "blacklist = itoken

0 Karma

dnavia29
New Member

No I am not using a deployment server, I am trying this one in QA first

0 Karma

richgalloway
SplunkTrust
SplunkTrust

We shouldn't make changes to etc/system/local on a UF. It prevents the deployment server from overriding those changes later.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

codebuilder
Influencer

@richgalloway that is a very valid point. I was just trying to provide a quick example, but you are definitely correct.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

dnavia29
New Member

Yes that's right, the inputs.conf I found it "/opt/splunkforwarder/etc/search/local" so that way prevents what you just said... Thank you, I think that might work.. I need to wait now to see if it's not logging anymore

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Blacklists are the usual way to do that. Why can you not use them?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

dnavia29
New Member

Because I see in the documentation and Saw this "Add the following line to your monitor stanza in the /local/inputs.conf file for the app context that you defined the input in.
blacklist = "

I am not sure if this it has to be done in the inputs.conf of the SplunkUniversalForwarder, and try to to a regular expression that excludes this one "itoken-cl-app.log" for example.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, blacklist goes in the inputs.conf file on the UF. I don't understand how that prevents you from specifying a blacklist.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!