Getting Data In

Index masking issue on splunk enterprise for iis logs

Path Finder

Hi,
We have authentication session id field from IIS logs needs to be masked on top priority due to high security standard. In my environment, we have 2 indexers, 1 search head and 2k universal forwarders existed.

We have Splunk enterprise 6.6.3 version.

I have done the masking on 2 indexers in props.conf and transforms.conf file. Below respective stazas & log entries. I need to mask the AUTH_SESSION_ID value below.

AUTH_SESSION_ID=SMS_archprod310_71430f::91a02ede5b238d00fedba55284bb7668

props.conf
[iis]
TRANSFORMS-anonymize = session-anonymizer

transforms.conf
[session-anonymizer]
REGEX = (?m)^(.)AUTH_SESSION_ID=.(\s-\ss\d{4}srs\d{3}.*)$
FORMAT = $1AUTH_SESSION_ID=########$2
DEST_KEY = _raw

Below are the sample iis logs:

2018-04-14 00:23:36 W3SVC1 s9999srs999. GET /RESTStoreServices/RESTPricing.svc/GetDefectiveInfo/1980/618786551281/UPC - AUTH_SESSION_ID=SMS_testtest310_71430f::8db899cf4a52c5d6efb43d9ba34d6190 - s9999srs999.company.com

The above props.conf and transforms.conf is perfectly working fine when I ingest the log entries manually using splunk search UI. However, logs coming from Universal forwarders are not masking. I did restart the indexer services as well. But no use.

As this is critical to us, kindly provide the resolution ASAP.

Thanks,
Ramu Chittiprolu

0 Karma

Champion

Can you try putting SEDCMD directly in props.conf as follow:

[iis]
SEDCMD-abc = s/AUTH_SESSION_ID=(.*)/AUTH_SESSION_ID=XXXXX/g
0 Karma

Path Finder

Removed the old props.conf and transforms.conf configuration. Added the below line in props.conf

==========
[iis]

SEDCMD-abc = s/AUTH_SESSION_ID=(.*)/AUTH_SESSION_ID=XXXXX/g

I still see the below entry in search logs. Masking is not done. Restarted the indexer services as well.

2018-04-14 09:38:44 W3SVC1 GET /wireless/storesubmenu/storesubmenu.aspx devicename=TC70 80 - 10.145.133.153 HTTP/1.1 Mozilla/5.0+(Linux;+U;+Android+2.2;+en-us;+Nexus+One+Build/FRF91)+AppleWebKit/533.1+(KHTML,+like+Gecko)+Version/4.0+Mobile+Safari/533.1 AUTH_SESSION_ID=SMS_test310_71430f::3279gat681ffdfd78961a7a9f4fd40fb1 - s9999srs001.company.com 200 0 30486 545 182

0 Karma

Champion

Did you restart Splunk service?

0 Karma

Champion

Also if you want to anonymize only session id change regex to :

[iis]
SEDCMD-abc = s/AUTH_SESSION_ID=([^/s]*)/AUTH_SESSION_ID=XXXXX/g
0 Karma

Path Finder

p_gurav
Did you restart Splunk service?

Which Splunk service are you referring to?

If you are asking about Splunk Indexer services restart, I did that. But still masking is not happening. I have 2 indexer servers, restarted the services in both. No luck.

0 Karma

Champion

If you are using indexer clustering, did you put props in master apps on master server and then push to indexers?

0 Karma

Path Finder

we don't have index clustering. It's only load balancing and updated props.conf in all indexers. Not sure what is happening, tried in all the ways.

Do i have anything to do at the Universal forwarder side or Deployments server or Search side server side ?

There is something wrong, The same SED expression OR props.conf/transforms.conf is perfectly working fine in my local test splunk server. But in production it's not working 😞

0 Karma

Champion

Try putting props in forwarder and also verify sourcetype.

0 Karma

Path Finder

Verified sourcetype. Sourcetype already exists in indexer /etc/apps/dir/inputs.conf.

But there is no document suggest that props.conf needs to be updated at universal forwarder side?

0 Karma