IIS logs :Need to mask cs_cookie,cs_Referer and cs_uri_path but headers still showing values after using SEDCMD

[sag5757]

Need to mask cs_cookie,cs_Referer and cs_uri_path but headers still showing values after using SEDCMD.i need to mask the header values as well. 

[isoutamo]

Hi

If you are doing this at  indexing time then add to the transforms.conf on the first full Splunk Enterprise instance from UF to Indexer(s) 

INGEST_EVAL-cs_cookie = null() 

under correct stanza.

And if this is a search time then add to SH's props.conf 

EVAL-cs_cookien = null()

And same for all those fields.

r. Ismo

[sag5757]

 I have added  below  add to SH's props.conf . it still not working

[iis]

BREAK_ONLY_BEFORE = ^\d{7}
BREAK_ONLY_BEFORE_DATE =
CHECK_FOR_HEADER = true
DATETIME_CONFIG =
EVAL-action = "allowed"
EVAL-app = "iis"
EVAL-http_content_type = "text/html"
EVENT_BREAKER = .+
INDEXED_EXTRACTIONS = w3c
LINE_BREAKER = ([\r\n]+)\d+-\d+-\d+\s\d+:\d+:\d+
MAX_TIMESTAMP_LOOKAHEAD = 32
NO_BINARY_CHECK = true
SEDCMD-cs_Cookie = s/(?<cs_Cookie>[\w]+=[^\s]+)/*******/g
SEDCMD-cs_Referer = s/(?<cs_Referer>[\w]+:\/\/[^\s]+)/*******/g
SEDCMD-cs_uri_query = s/(?<cs_uri_query>\w+=[^\s]+|(\w+=))/*******/g
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TIME_PREFIX = ^
TRUNCATE = 0
TZ = UTC
category = Custom
description = W3C Extended log format produced by the Microsoft Internet Information Services (IIS) web server
detect_trailing_nulls = auto
disabled = false
pulldown_type = true
EVAL-cs_Cookie = null()
EVAL-cs_Referer= null()
EVAL-cs_uri_query= null()

[isoutamo · Accepted solution]

Actually you cannot use SEDCMD on SH it's working only in indexing time (see https://docs.splunk.com/Documentation/Splunk/8.0.5/Admin/Propsconf) !

If those SEDCMDs are running on IDX layer then those define cs_XXX fields as indexed fields. If you want to get ride of those fields then remove "?<cs_Cookie>" etc. part from SEDCMD and leave just masking part there like

SEDCMD-cs_Cookie = s/([\w]+)=([^\s]+)/\1=*******/g

When you want do those on search time you must use props.conf to the correct sourcetype.

props.conf

[iis]
EVAL-_raw = replace(_raw, "([\w]+)=([^\s]+)","\1=\"*******\”")
EVAL-cs_Cookie = null()
EVAL-cs_Referer= null()
EVAL-cs_uri_query= null()
....

As you could have only one EVAL-<fieldname> (like _raw) you must fix previous regex to match all events which you want to mask.

A good tool to check what props and transforms will match your queries is to use btool

splunk btool props list iis --debug which shows you what it try to do and where it get those expressions.

But much better approach is that some one fix the logs that there haven't been this kind of events and if this is not possible then second option is masks those on indexing phase. To be hones there is no way how to mask those event reliability on search time so that anyone cannot see those!

r. Ismo