topic Re: Not able to mask the data.. in Getting Data In

Not able to mask the data..

vikcee — Fri, 25 Oct 2019 06:28:54 GMT

Hi,

For my learning purpose, I have installed splunk and configured universal forwarder. Now I want to Hide/mask some data. But not able to do, Below are the required details.
Please let me know anything else is needed.

Sample log
SERVICE NOTIFICATION: 9123456780;www.test.com;Kibana_Service_Check;CRITICAL;notify-service-by-email;connect to address xx.xx.xxx.xxx and port 5601: Connection refused

Sample Mobile Number(9123456780) need to masked

Filed details:

Contact_group=9123456780

props.conf

[nagios]
TRANSFORMS-anonymize = Contact_group-anonymizer

transforms.conf

[Contact_group-anonymizer]
REGEX = (?m)^(.*)Contact_group=^[\[\]\d\s\w]+.\s(?<Contact_group>[a-z]+).
FORMAT = $1Contact_group=###########$2
DEST_KEY = _raw

Re: Not able to mask the data..

FrankVl — Wed, 30 Sep 2020 02:45:07 GMT

Where did you deploy this masking config? This should be on a full Splunk Enterprise instance, not on the UF.

Also: your regex looks for 'Contact_group=', while that string does not exist in your raw event. Also not sure what that ^ is doing there, behind the 'Contact_group='.

So I think your regex needs some work. Try tools like regex101.com to test your regex and see if it captures what you need.

Also: this may be done much simpler and much more efficiently by using a SEDCMD in props.conf.

Re: Not able to mask the data..

gcusello — Fri, 25 Oct 2019 08:19:14 GMT

Hi
you have to modify
props.conf

[your_sourcetype]
TRANSFORMS-anonymize = session-anonymizer

transforms.conf

 [session-anonymizer]
 REGEX = ^SERVICE NOTIFICATION:\s[^;]*(.*)
 FORMAT = ^SERVICE NOTIFICATION:\s########,$1
 DEST_KEY = _raw

you can test regex at https://regex101.com/r/WNni5C/1 .

For more info, see at https://docs.splunk.com/Documentation/Splunk/8.0.0/Data/Anonymizedata

Ciao.
Giuseppe

Re: Not able to mask the data..

vikcee — Fri, 25 Oct 2019 08:55:42 GMT

Hello Sir,

Thanks for the quick response.

I have added your comment as it is in props.conf and transform.conf,but its not working.

below is the log file.

[1571992954] SERVICE NOTIFICATION: nagiosadmin;www.test.com;Elastic_Service_Check;CRITICAL;notify-service-by-email;connect to address xx.xx.xxx.xxx and port 9200: Connection refused

for my case nagiosadmin or 9123456780 are the filed with filed name "Contact_group". does this has anything to do.

And I have one more question: In regex section what exactly is needed?

thanks
Vikash

Re: Not able to mask the data..

vikcee — Fri, 25 Oct 2019 09:01:52 GMT

Hello Sir,

I am very new to Splunk. I have deployed the above conf in props.config and transform.conf. I am not sure about the Regex section what exactly need to be done. Do we need to write regex to fetch the filed that we are planning to mask or something else.

https://regex101.com/r/7roqEj/1

And thanks for the suggestion about SEDCMD, I will check the document, How can I do this.

Thanks
Vikash

Re: Not able to mask the data..

gcusello — Fri, 25 Oct 2019 09:12:53 GMT

HI vikcee,
section is a part of the source that must remain as original, in you sample: the first part must remain as original, the central part must bu masked, the second section must remain as original.

The sample now is different, use this regex in transforms.conf:

REGEX = ^(.*)SERVICE NOTIFICATION:\s[^;]*;(.*)
FORMAT = ^$1SERVICE NOTIFICATION:\s########;$2

You can test regex at https://regex101.com/r/WNni5C/2

Ciao.
Giuseppe

Re: Not able to mask the data..

kartm2020 — Fri, 25 Oct 2019 09:41:01 GMT

Hi Vikcee,
Please modify as like below
props.conf

[your_sourcetype]
TRANSFORMS = session-anonymizer
transforms.conf

[session-anonymizer]
DEST_KEY = _raw
REGEX = ^SERVICE NOTIFICATION:\s\d+
FORMAT = $1xxxxxxxxxx

To answer your question.
By default splunk will extract the field. Sometimes splunk would n't able to extract the field. So we need to write a regex to extract the field whichever we want. In this scenario, we must need to write to regex to tell splunk which wants to be masked.

Re: Not able to mask the data..

woodcock — Fri, 25 Oct 2019 18:34:02 GMT

Try this on your indexers:

props.conf

[nagios]
SEDCMD-anonymize_Contact_group = s/SERVICE NOTIFICATION:\s+\d+/SERVICE NOTIFICATION: Contact_group=###########/

Re: Not able to mask the data..

vikcee — Tue, 29 Oct 2019 10:15:01 GMT

Hello sir,

Thanks a lot. Its working as expected. But again one more question. If I have to mast partially. Such as last 5 characters or 5 characters from start.

Re: Not able to mask the data..

gcusello — Tue, 29 Oct 2019 10:34:08 GMT

You're welcome!
Anyway, the approach is the same, you have only to change the regex:

REGEX = ^(.*)SERVICE NOTIFICATION:\s[^;]{5}(.*)
FORMAT = ^$1SERVICE NOTIFICATION:\s#####;$2

In this way only the first 5 chars of the number or of the user are masked and the other are showed.
You can test it at https://regex101.com/r/WNni5C/3 .

Ciao and Next time!
Giuseppe

Re: Not able to mask the data..

vikcee — Tue, 29 Oct 2019 11:31:01 GMT

Hello Sir,

Got the point..Thanks A lot.

For my practice I have masked all the field one by one 😛

Thanks
Vikash

Re: Not able to mask the data..

gcusello — Tue, 29 Oct 2019 11:41:56 GMT

Happy to helped you!
Ciao and Next Time!
Giuseppe