Splunk Search

Can you help me mask some data during search time?

impurush
Contributor

Hi Splunkers,

I want to mask the PII data during the search time for specific users.

I checked all the existing questions and answers, but was not able to achieve it. I tried the below option,

  1. I masked the data using the below search, and I saved as a macro, and then, I tried to give access to the user for only this macro, but the macro did not run. my search|rex field=_raw "(?.*)TYPE\s\[PHONE\]\s*\[\+\w{12}\](?.*)" | eval _raw=head."TYPE [PHONE] [############] ".tail
  2. I tried to use regex in props/transforms in the search head, but it did not help.
  3. I don't want re-index the data because the volume is large.
  4. I tried to do event type, but the query cannot have the pipe symbol.

Any help is appreciated.

Thanks,
Purush

0 Karma

somesoni2
SplunkTrust
SplunkTrust

YOu can find the search time data masking methods here (see accepted answer and comment below it)
https://answers.splunk.com/answers/234919/search-time-data-masking.html

The problem is that since it's done by search time, underlying raw data still have those PII data. So, anyone can run a basic search with "Fast Mode" to disable this masking and see the original data. In cases like this, we do one of following (along with working with owner to mast the PII at the source OR do the mask at index time):

1) Delete the current data with PII and re-index it. (causes duplicate license usage)
2) Move the to a summary index (using summary index search OR using collect command), with the summary search doing search time masking. Once moved, delete the original PII data.

0 Karma

impurush
Contributor

Thank you for your answer!

  1. I don't want to re-index because of large volume. For Summary Indexing, no need to re-index, just run the job and save the metrics to the summary index and do not give the access to the original index., that way we restricted the user to see the original data. Due to too many types of data and jobs, we are not moving towards it.

  2. I agreed that if we save as a macro and if the user knows the base query, still he/she can see the data.
    For this option, I tried to restrict the access to the only Macro but it didn't work.

0 Karma

Vijeta
Influencer

Is your rex command working fine?

0 Karma

impurush
Contributor

yes, perfectly working in all the options. I verified with regex online editor also. The problem with the first option is that I am able to give macro in the restrict search terms for a role but when I search as a user belongs to that role, no data is populating.

0 Karma

adonio
Ultra Champion

you need to use mode=sed
see here:
https://docs.splunk.com/Documentation/Splunk/7.2.5/SearchReference/Rex

also, you probably would like to add it as a search filter to the role the users belong too.

note: imho its not a sustainable solution. better way would be to put masked data in a summary index and allow users to see summarized results only

impurush
Contributor

Yes, Summary indexing is the best solution which we have already implemented for one part of data but the problem with summary indexing is that you need to run a job for every 5 mins to make it lively and I have different types of data and our alerts are running extensively, so if we run the summary indexing job every 5 mins which may not be effective for our environment.

PS: I am going to think in this direction of how much efficiently I can use SI in our environment.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...