Getting Data In

How to filter certain events so they are not returned?

Explorer

Our team is trying to filter out events that occur with certain tags in them. For example:

[19/Mar/2013:23:59:57 -0400] "GET /favicon.ico HTTP/1.1" 404 10607 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0

I want to Splunk to not return any logs with "favicon.ico" in it. How would we go about doing this? Our goal is to have Splunk only return data that is relevant to finding issues and not the data we consider junk. Much thanks!

Tags (3)
1 Solution

Champion

This can take place on the indexer or Heavy Forwarder using a props.conf and tranforms.conf. Will prevent any event contain the GET\s/favicon.ico from being indexed at index time.

transforms.conf


[removefavicon]
REGEX = GET\s/favicon.ico
DEST_KEY = queue
FORMAT = nullQueue


#iisw3c is is the source type of your IIS weblogs. Though you many have change that to IIS or something else to match your sourcetype.
[iisw3c]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = False
TZ = GMT
REPORT-iisw3cfields = iisw3cfields
#Entry below is what you really want
TRANSFORMS-removefavicon = removefavicon

Alternativly you could use REPORT-<yourtransform> = <yourtransform> settings to remove it from search time.

Hope this helps or gets you started. dont forget to accept answers or vote them up if they help.

my other post:
how-to-index-w3c-iislog-from-a-universal-forwarder

View solution in original post

Champion

This can take place on the indexer or Heavy Forwarder using a props.conf and tranforms.conf. Will prevent any event contain the GET\s/favicon.ico from being indexed at index time.

transforms.conf


[removefavicon]
REGEX = GET\s/favicon.ico
DEST_KEY = queue
FORMAT = nullQueue


#iisw3c is is the source type of your IIS weblogs. Though you many have change that to IIS or something else to match your sourcetype.
[iisw3c]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = False
TZ = GMT
REPORT-iisw3cfields = iisw3cfields
#Entry below is what you really want
TRANSFORMS-removefavicon = removefavicon

Alternativly you could use REPORT-<yourtransform> = <yourtransform> settings to remove it from search time.

Hope this helps or gets you started. dont forget to accept answers or vote them up if they help.

my other post:
how-to-index-w3c-iislog-from-a-universal-forwarder

View solution in original post

Explorer

Before indexing since we want the filter to be permanent as we find more stuff to filter out.

0 Karma

Explorer

Add "NOT favicon.ico" to the search string or Alt-click on the tag you don't want to have Splunk do that for you.

0 Karma

Explorer

I need something that would essentially be permanent for when I go to review logs. I need each tag that we do not need to not show up at all.

0 Karma

Champion

Do you want to do this before or after indexing?

0 Karma