topic Re: Filter Client IP from IIS 7.5 Logs in Getting Data In

Filter Client IP from IIS 7.5 Logs

rmsit — Tue, 29 Sep 2020 09:14:24 GMT

Hello, all.

I would like to filter out a specific client IP address from my IIS logs. What would be the best approach, have the UF or indexer perform the filtering and how do I go about doing that? Configuration for indexer below. Thanks! - James

Props.conf
[iis]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = False
TZ = GMT
REPORT-iisw3cfields = iisw3cfields
TRANSFORMS-removecomments = removecomments

Transforms.conf

IIS W3C Log field extractions (Identical in IIS 6 and 7)

These assume that you have enabled all available fields to be logged

[iisw3cfields]
DELIMS = " "
FIELDS = date,time,s_sitename,s_computername,s_ip,cs_method,cs_uri_stem,cs_uri_query,s_port,cs_username,c_ip,cs_version,cs(User_Agent),cs(Cookie),cs(Referer),cs_host,sc_status,sc_substatus,sc_win32_status,sc_bytes,cs_bytes,time_taken

Re: Filter Client IP from IIS 7.5 Logs

lguinn2 — Fri, 01 Apr 2016 18:07:09 GMT

It is generally best to filter at the indexer, because you want to keep the forwarder as lightweight as possible (since it is usually running on a production server). But you can't filter on a universal forwarder anyway - it doesn't do parsing. All of the filtering settings must go on the indexer, as filtering happens during parsing time.

Add to props.conf

TRANSFORMS-filter1 = filter_ip

Add to transforms.conf

[filter_ip]
REGEX = 192\.168\.1\.172
DEST_KEY = queue
FORMAT = nullQueue

You could make the regular expression more specific; in my example, it is looking for an IP address that appears anywhere in the event.

BTW, I notice that you have TRANSFORMS-removecomments = removecomments in your props.conf, but no corresponding stanza in transforms.conf...

Re: Filter Client IP from IIS 7.5 Logs

rmsit — Tue, 29 Sep 2020 09:17:34 GMT

Thanks so much. So if I wanted to drop a specific IP addresses from field "c_ip", I would specify this in transforms.conf? How could I specific multiple IP addresses not on the same subnet?

[filter_ip]
REGEX = c_ip=192.168.1.254
DEST_KEY = queue
FORMAT = nullQueue

Re: Filter Client IP from IIS 7.5 Logs

rmsit — Fri, 01 Apr 2016 20:04:34 GMT

I may have answered by own question. I will see if this works. Thanks again.

REGEX = c_ip=("192.80.134.59|192.67.213.194|192.67.213.166")

Re: Filter Client IP from IIS 7.5 Logs

lguinn2 — Fri, 01 Apr 2016 20:37:09 GMT

You should not include the field name c_ip

At this point in the parsing process, Splunkcan't use the field names. So it should be

REGEX = 192.80.134.59|192.67.213.194|192.67.213.166

Or maybe there is a way... but it won't be using the regex that you showed. I'll look it up...

Re: Filter Client IP from IIS 7.5 Logs

montgomeryam — Fri, 30 Sep 2016 16:48:36 GMT

@lguinn Did you by chance have a chance to look this up and determine how to filter by field names?

I am trying to do the same, but not having any luck.