topic Re: Index time masking maintaining string length in Getting Data In

Index time masking maintaining string length

payl_chdhry — Tue, 18 Aug 2020 13:55:40 GMT

Hi,

I want to do masking for logs at index time but the replaced value ("X" here) should be same character length as original string. My requirement for masking is that for the value within [] only last 4 characters should be visible.

Example for below logs:

2020-08-18T13:17:43,990 [Engine 1] TRACE log data V01 [1|12345678]
2020-08-18T13:17:44,979 [Engine 2] TRACE log data V02 [2|A35453DFDF65]

The indexed logs should be:

2020-08-18T13:17:43,990 [Engine 1] TRACE log data V01 [1|XXXX5678]
2020-08-18T13:17:44,979 [Engine 2] TRACE log data V02 [2|XXXXXXXXDF65]

Currently I am using SEDCMD command as below but that is taking only static length for X:
s/(TRACE\slog\s+data\s+V\d+\s+\[\d\|)(\w+)(\w{4})/\1XXXXXX\3/g

Is there a way to replace string at index time with another string maintaining the count of characters.

Re: Index time masking maintaining string length

thambisetty — Tue, 18 Aug 2020 19:30:14 GMT

Is Length of the string dynamic?

Re: Index time masking maintaining string length

cpetterborg — Wed, 19 Aug 2020 00:28:13 GMT

Using the clever regex from @thambisetty (and slightly modified), here is a SEDCMD that I have tested which works and does what you want:

SEDCMD=s/(?=[^\|]+\w{4}]$)./#/g

Re: Index time masking maintaining string length

payl_chdhry — Wed, 19 Aug 2020 04:11:20 GMT

@thambisetty yes the length would be dynamic. It should be anywhere between say 8 to maybe 20.

Re: Index time masking maintaining string length

to4kawa — Wed, 19 Aug 2020 04:12:39 GMT

SEDCMD-mask = s/\|(\w+)(\w{4}\])$/|#\2/

Why can't we use this one, which has less of a load?

Re: Index time masking maintaining string length

payl_chdhry — Wed, 19 Aug 2020 04:13:17 GMT

great. Let me try this. @cpetterborg

Re: Index time masking maintaining string length

thambisetty — Wed, 19 Aug 2020 04:28:27 GMT

No worries 😉, I shared tested regex.

upvote if that solves your problem.

Re: Index time masking maintaining string length

payl_chdhry — Wed, 19 Aug 2020 07:32:42 GMT

@to4kawa that would replace the whole matching string with single #, like below:

2020-08-19T07:28:32,032 [Engine 8] TRACE SegmentLocks V13 added [1|A454FFDPP01]

To:

2020-08-19T07:28:32,032 [Engine 8] TRACE SegmentLocks V13 added [1|#PP01]

My requirement is to replace each character (and not whole string) with X so the count of characters replaced before masking is equal to count of Xs.

2020-08-19T07:28:32,032 [Engine 8] TRACE SegmentLocks V13 added [1|XXXXXXXPP01]

Re: Index time masking maintaining string length

payl_chdhry — Wed, 19 Aug 2020 07:45:42 GMT

@cpetterborg Thank you for this. It worked 🙂

I had one more query in case you might be able to help. I have same requirement but for different format of string. I am trying to customize this rex/sed for this format as well but not able to achieve it yet.

2020-08-19T07:42:38,942 [Engine 9] TRACE MEHSegment WHERE "00" "00000123456 " 1 240

should give me:

2020-08-19T07:42:38,942 [Engine 9] TRACE MEHSegment WHERE "00" "0000012XXXX " 1 240

Re: Index time masking maintaining string length

thambisetty — Wed, 19 Aug 2020 08:01:46 GMT

@payl_chdhry
you could upvote my post also, because my post is the source to your answer.😉

Re: Index time masking maintaining string length

payl_chdhry — Wed, 19 Aug 2020 08:19:46 GMT

Sorry m new to posting here. How do I upvote a post once I have marked anther post as answer.

Re: Index time masking maintaining string length

thambisetty — Wed, 19 Aug 2020 09:38:03 GMT

Can you post this as new question?

Re: Index time masking maintaining string length

payl_chdhry — Wed, 19 Aug 2020 10:08:59 GMT

just did 🙂
https://community.splunk.com/t5/Getting-Data-In/Index-time-masking-maintaining-string-length-continue/m-p/514890#M87226