Splunk Search
Highlighted

How to omit categories of log entries

Engager

When we are browsing log files for problems, we often don't know exactly what we're looking for. But in a short period of time, we often know what we don't want to look at. This is especially true when browsing thru Data Power log files with the DEBUG log option turned on. Here is an example of what I DON'T want to see:

Jun 24 16:51:52 DP-101 [EP1350][http][error] xmlfirewall(ImageUpload1_XFW): trans(8194384)[20.20.138.135]: could not establish SSL for incoming connection
Jun 24 16:51:52 DP-101 [EP1350][http][error] xmlfirewall(ImageUpload1_XFW): trans(8194384)[20.20.138.135]: could not establish SSL for incoming connection
Jun 24 16:51:51 155.82-default [mpgw][error] trans(10646082)[12.23.28.201]: Request processing failed: Connection terminated before request headers read

These are all "pings" from a network firewall checking to see if a port is ready to receive data. I don't want to see these.

Any ideas on how to omit them?

Tags (2)
0 Karma
Highlighted

Re: How to omit categories of log entries

Champion

Based on the sample provided, you can omit these events from your search results by appending your search string:

search = myoriginal search NOT source=/nfsin/enysunadm001/*/messages ImageUpload1_XFW 

If this omits too many events you'll probably want to find a search to return only events that you don't want to see and create an eventtype:

http://www.splunk.com/base/Documentation/latest/User/ClassifyAndGroupSimilarEvents

Once you create your eventtype, let's call it eventtypetag=firewallping, you can append the following to whatever search you conduct to omit these events:

search = mysearch terms NOT eventtype=firewallping
Highlighted

Re: How to omit categories of log entries

Super Champion

You can alt-click on terms you would like to omit from your search. For example, you could alt-click on the term ImageUpload1_XFW and that may eliminate what you want. Obviously you'll have to play around and try a few different options to get something that works well with your data.

Keep in mind that you can search for things or search to exclude things. Here are a few different ideas to add to your search command:

Here is a search for warnings errors and failures that excludes the two example you've given. (but other errors will show up)

(warn* OR error OR fail*) NOT ("xmlfirewall" "establish SSL" "incoming connection") NOT (mpgw "Connection terminated before request headers read")

You can continue to add additional "NOT" expressions until you filtered out your unwanted events.

Be sure to check out the docs page linked to by the wolverine

0 Karma
Highlighted

Re: How to omit categories of log entries

SplunkTrust
SplunkTrust

Lowell - it really should be ALT-click. If you're finding that it's CTRL-click on your system that is a bug. CTRL-click should actually replace the entire search with the term you clicked on. ALT-click will add the negation of the term you clicked on.

0 Karma
Highlighted

Re: How to omit categories of log entries

Super Champion

Thanks Nick. Your are correct, I've updated the answer.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.