Deployment Architecture

Unable to anonymise data even after we have right regex.

vikram_m
Path Finder

I have a log file as detailed below.

"02d4224f-8cc8-48b4-a06d-034a97ad68a8-333999","2018-03-24T04:58:09.280Z","OPENIG-HTTP-ACCESS","02d4224f-8cc8-48b4-a06d-034a97ad68a8-333998",,,"10.255.0.19","8443","10.255.0.11","11700",,,,"true","GET","https://imsdat.sentienceanalytics.com:443/sentience/iams/api/auth/services/system","{}","{""activity... eyAidHlwIjogIkpXVCIsICJraWQiOiAiaFNsNTU4b2NrZHpsWmFHTXpweUZUOGZSamg0PSIsICJhbGciOiAiUlMyNTYiIH0.eyAiYXRfaGFzaCI6ICJIemJPcTJkLTB5LThIWThMOS1ZdFVnIiwgInN1YiI6ICI5MDFkMTMwNy00NjdkLTRmZDctOTEzMC1lNzUwM2U5OGFhOWEiLCAiYXVkaXRUcmFja2luZ0lkIjogIjkyMDk5NzcyLWU0MWEtNGQwYi1hYmM1LTg2NzY3MGIzYTIyNC01NTg2NDQiLCAiaXNzIjogImh0dHBzOi8vaW1zZGF0LnNlbnRpZW5jZWFuYWx5dGljcy5jb206NDQzL29wZW5hbS9vYXV0aDIvY2JwZ2ZiZGV2IiwgInRva2VuTmFtZSI6ICJpZF90b2tlbiIsICJhdWQiOiAiOTAxZDEzMDctNDY3ZC00ZmQ3LTkxMzAtZTc1MDNlOThhYTlhIiwgIm9yZy5mb3JnZXJvY2sub3BlbmlkY29ubmVjdC5vcHMiOiAiZDg5NTM5ZTUtNDA3Mi00MDNkLWExYjEtM2Q2M2FkZmI5ZmVjIiwgImF6cCI6ICI5MDFkMTMwNy00NjdkLTRmZDctOTEzMC1lNzUwM2U5OGFhOWEiLCAiYXV0aF90aW1lIjogMTUyMTg2NzA2MSwgInJlYWxtIjogIi9jYnBnZmJkZXYiLCAiZXhwIjogMTUyMTg3MDY2MSwgInRva2VuVHlwZSI6ICJKV1RUb2tlbiIsICJpYXQiOiAxNTIxODY3MDYxIH0.WO7PoFhmieqV0tK03Zft0gUY2fDzd6-h0FJQtCoAWtUbiK_5v2OY-wbpXyMK3GC_XlOX4VfbavjBE4K00ryzhZsaQ2CZA-uQGgeDELQEsxk_yNQyex00WpPpq0Xg8S_4gNBDg3HvfrlDrGD6MYoWRhrP2K3i6ggbSqdCPu5YSqIRF9P06WqeoD01z4tm4D_4tG-Z17_EuXllUAmtoPqN0ZKrYda56HsS-u6EuRqHvUViHIeawCwTgzdkvGMzvWs-SoP5-_djHUP7mZbI2I4CoS_eDTEbTiJ8uYOyvhXnLEqmOjy4FYIHHqP6E4yA2bGDfVCtj85ixOEZfxLCp_yG9g""],""host"":[""imsdat.sentienceanalytics.com""],""user-agent"":[""Honeywell.Sentience.WebApiClient.Identity.Rest.SentienceIdentityPublicWebApis/1.0.0-ci.182+Branch.master.Sha.0a4c3e515876f9ce616d0bf07bd1fe9bb7831f6c""]}","{}","{""Cache-Control"":[""no-cache, no-store, max-age=0, must-revalidate""],""Content-Type"":[""application/json; charset=UTF-8""],""Date"":[""Sat, 24 Mar 2018 04:58:09 GMT""],""Expires"":[""0""],""Pragma"":[""no-cache""],""RequestId"":[""02d4224f-8cc8-48b4-a06d-034a97ad68a8-333998""],""X-Content-Type-Options"":[""nosniff""],""X-Frame-Options"":[""DENY""],""X-XSS-Protection"":[""1; mode=block""]}","SUCCESSFUL","200",,"18","MILLISECONDS"

When I wrote the SED command in Linux and checked I am able to see the Bearer field is hashed.

[splunk@splunkqadeployer ~]$ sed 's/Bearer (.{1150})/#####/g' /tmp/test.anon
"02d4224f-8cc8-48b4-a06d-034a97ad68a8-334002","2018-03-24T04:58:09.313Z","OPENIG-HTTP-ACCESS","02d4224f-8cc8-48b4-a06d-034a97ad68a8-334001",,,"10.255.0.19","8443","10.255.0.11","11700",,,,"true","POST","https://imsdat.sentienceanalytics.com:443/sentience/iams/api/auth/systems/8a5bc996-8045-41af-b137-71...; charset=utf-8""],""host"":[""imsdat.sentienceanalytics.com""],""user-agent"":[""Honeywell.Sentience.WebApiClient.Identity.Rest.SentienceIdentityPublicWebApis/1.0.0-ci.182+Branch.master.Sha.0a4c3e515876f9ce616d0bf07bd1fe9bb7831f6c""]}","{}","{""Cache-Control"":[""no-cache, no-store, max-age=0, must-revalidate""],""Content-Length"":[""0""],""Date"":[""Sat, 24 Mar 2018 04:58:09 GMT""],""Expires"":[""0""],""Pragma"":[""no-cache""],""RequestId"":[""02d4224f-8cc8-48b4-a06d-034a97ad68a8-334001""],""X-Content-Type-Options"":[""nosniff""],""X-Frame-Options"":[""DENY""],""X-XSS-Protection"":[""1; mode=block""]}","SUCCESSFUL","200",,"75","MILLISECONDS"

I had put this thing into props.conf with below parameters in place.

[splunk@splunkqadeployer local]$ cat props.conf
[SOURCETYPE]
SEDCMD-Bearer = s/Bearer (.{1150})/#####/g

I deployed this app into the indexer and UF from where these files are comming.

after multiple efforts I am unable to see the data masked. May I know where I am missing here. As RegEx seems fine by the app deployment is getting failed for me.

0 Karma

mayurr98
Super Champion

you need to put this in indexer or Heavy forwarder mostly in splunk/etc/apps/<app-name>/local.
If you have indexer-cluster then put this configuration in master-apps/<appname>/props.conf on the cluster master and then push the bundle from the cluster master GUI.
This configuration will apply to latest & future events and it will not apply to historical events. So check latest events only.
let me know if this helps!

0 Karma

vikram_m
Path Finder

Thanks for your reply @mayurr98 the same configuration is deployed which you have said above.

can you confirm if the sourcetype definition which we are doing is correct or do we need to have a bracket like <> when we want to define our sourcetype.

0 Karma

aakwah
Builder

Universal forwarder can not perform this parsing you have to use you need to put this in Heavy forwarder as mentioned in the above comment.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...