Getting Data In
Highlighted

Data masking using heavy forwarders

Contributor

Been trying to mask data before indexing into indexer using heavy forwarders. below is the log sample and data am trying to mask

JSESSIONID=SD1SL10FF3ADFF3" to JSESSIONID=#######FF3ADFF3"

189.222.1.46 - - [24/Jul/2014:11:27:00] "GET /flowerstore/product.screen?productid=RP-SN-01 HTTP/1.1" 200 10897 "http://mystore.splunk.com/flower_store/category.screen?category_id=BALLOONS&JSESSIONID=SD1SL10FF3ADF..." "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 527 3006

10.2.91.38 - - [24/Jul/2014:11:28:00] "POST /flowerstore/jsignoncheck HTTP/1.1" 302 309
"http://mystore.splunk.com/flower
store/enterorderinformation.screen&JSESSIONID=SD1SL10FF3ADFF3" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 3441 2576

192.0.1.38 - - [24/Jul/2014:11:28:15] "GET /flowerstore/images/cat3.gif HTTP/1.1" 200 5024 "http://mystore.splunk.com/flowerstore/item.screen?item_id=EST-21&JSESSIONID=SD1SL10FF3ADFF3" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 4323 3071

below is my props.conf and transforms.conf:

[access_common]
TRANSFORMS-anonymize = session-anonymizer

[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw

Kindly advice....i do not see and changes in fields after applying this configuration

0 Karma
Highlighted

Re: Data masking using heavy forwarders

SplunkTrust
SplunkTrust

Hi @Venkat_16,

Please use below configuration in transforms.conf on Heavy forwarder.

[session-anonymizer]
REGEX = (?m)^(.*)JSESSIONID=.*((?=\"\s\").*)$
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw

After changing above configuration please reload configuration using https://<HF FQDN>:8000/debug/refresh OR restart splunk on heavy forwarder.

I hope this helps.

Thanks,
Harshil

Highlighted

Re: Data masking using heavy forwarders

Contributor

Thanks for your answer @harsmarvania57. But it didnt work 😞

[access_common]
TRANSFORMS-anonymize = session-anonymizer

[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw

i restarted heavy forwarder also....logs are getting indexed with out any masking

0 Karma
Highlighted

Re: Data masking using heavy forwarders

SplunkTrust
SplunkTrust

Looks like you are still using your old configuration, please use configuration which I have provided.

0 Karma
Highlighted

Re: Data masking using heavy forwarders

Contributor

sorry it was copy paste error:

[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=.((?=\"\s\").*)$
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw
is the one am using

0 Karma
Highlighted

Re: Data masking using heavy forwarders

SplunkTrust
SplunkTrust

Still this configuration is wrong, you can see my regex has (?m)^(.*)
and configuration which you pasted starts with (?m)^(.)

0 Karma
Highlighted

Re: Data masking using heavy forwarders

Contributor

Thanks alot for helping patiently but still it didnt work:

[access_common]
TRANSFORMS-anonymize = session-anonymizer

[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=.((?=\"\s\").*)$
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw

0 Karma
Highlighted

Re: Data masking using heavy forwarders

SplunkTrust
SplunkTrust

Please refer this https://regex101.com/r/CftIqK/1, regex is working perfectly fine. Can you please provide your configuration in Code Sample format because your REGEX is still wrong, you can see 101010 button when you type comment/answer please use that and paste your transforms.conf configuration.

0 Karma
Highlighted

Re: Data masking using heavy forwarders

SplunkTrust
SplunkTrust

All you need in this case is the following line in your props.conf file (no transforms.conf config needed) for the sourcetype:

SEDCMD-hidesessionid = s/JSESSIONID=\w{2}\d\w{2}\d{2}/JSESSIONID=#######/
0 Karma
Highlighted

Re: Data masking using heavy forwarders

Esteemed Legend

This works:

| makeresults 
| eval raw="189.222.1.46 - - [24/Jul/2014:11:27:00] \"GET /flower_store/product.screen?product_id=RP-SN-01 HTTP/1.1\" 200 10897 \"http://mystore.splunk.com/flower_store/category.screen?category_id=BALLOONS&JSESSIONID=SD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\" 527 3006:::10.2.91.38 - - [24/Jul/2014:11:28:00] \"POST /flower_store/j_signon_check HTTP/1.1\" 302 309 
\"http://mystore.splunk.com/flower_store/enter_order_information.screen&JSESSIONID=SD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\" 3441 2576:::192.0.1.38 - - [24/Jul/2014:11:28:15] \"GET /flower_store/images/cat3.gif HTTP/1.1\" 200 5024 \"http://mystore.splunk.com/flower_store/item.screen?item_id=EST-21&JSESSIONID=SD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\" 4323 3071"
| makemv delim=":::" raw
| mvexpand raw
| rename raw AS _raw
| rex mode=sed "s/JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)/JSESSIONID=#######\1/"

So you can use this:

 SEDCMD-session-anonymizer = s/JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)/JSESSIONID=#######\1/
0 Karma