topic Re: Search Time Masking Using Calculated Field in Splunk Search

Search Time Masking Using Calculated Field

arielpconsolaci — Thu, 28 Jan 2021 03:58:04 GMT

Hi Splunkers,

Good day. I am trying to perform search time masking using a Calculated Field to replace _raw with the required result.

This goes fine for me for my particular data of concern. However, it goes complex somehow when that particular field in the same event has to be masked another way. Citing an example below to explain more clearly.

Masking - 16 digits

2021/01/21 - 01:15 AM <ACT>1234567890123456</ACT>

Result: 2021/01/21 - 01:15 AM <ACT>123456######3456</ACT>

However, if I see 15 digits for this field, masking should be 5 ##### rather than 6 for 16digits.

Masking - 15 digits

2021/01/25 - 01:15 AM <ACT>987654321012345</ACT>

Result: 2021/01/25 - 01:15 AM <ACT>987654#####2345</ACT>

Since the same field, _raw, is being worked on. I reckon this is not possible.

props.conf

[<sourcetype>]

EVAL-_raw = replace(_raw,"(\d{6})(\d{5,6})(\d{4})","\1######\3")

Please let me know of your thoughts/suggestions.

Thanks in adv.

Cheers!

Re: Search Time Masking Using Calculated Field

manjunathmeti — Thu, 28 Jan 2021 05:07:34 GMT

hi, @arielpconsolaci,

You can use the EVAL command like below,

[<sourcetype>] EVAL-_raw = replace(_raw, "(\d{6})\d{5,}(\d{4})", "\1######\2")

If this reply helps you, an upvote/like would be appreciated.

Re: Search Time Masking Using Calculated Field

arielpconsolaci — Thu, 28 Jan 2021 03:57:03 GMT

Thanks @manjunathmeti .

I tried this. However, this does not work. Probably because it is the same field _raw.

I checked as well via btool and it only reads one of the eval calculations for the same sourcetype.

Re: Search Time Masking Using Calculated Field

to4kawa — Thu, 28 Jan 2021 04:25:38 GMT

| makeresults | eval _raw="2021/01/21 - 01:15 AM <ACT>1234567890123456</ACT> 2021/01/25 - 01:15 AM <ACT>987654321012345</ACT>" | multikv noheader=t | fields _raw | rex mode=sed "s/(\d{6})(\d{5,6})(\d{4})/\1######\3/g"

How about SEDCMD?

Re: Search Time Masking Using Calculated Field

manjunathmeti — Thu, 28 Jan 2021 05:06:59 GMT

Yeah, that is true. I've updated my answer. You can try it if the number of # doesn't matter in the replaced text.

Re: Search Time Masking Using Calculated Field

arielpconsolaci — Thu, 28 Jan 2021 07:25:55 GMT

Thanks again @manjunathmeti .

I am trying to observe that for 16 digits numbers be masked with 6#s between the first 6 digits and last 4 digits.

while for 15 digit numbers, masking should be just 5#s between the first 6 digits and last 4 digits.

Currently, with my current config, masking happens with 6#s. I am trying to get and observe 6#s and 5#s respective what is given (16 digit numbers and 15 digit numbers). From testing, this seems not possible. But let me know if otherwise.

Thanks for your time on this.

Re: Search Time Masking Using Calculated Field

arielpconsolaci — Thu, 28 Jan 2021 07:27:58 GMT

Thanks @to4kawa . That may work but what I am trying to achieve is that a plain search using the index and sourcetype at search time will return masked data accdg. to the rule set. Thus using a calculated field for _raw. Thanks still for the suggestion.