Index time masking maintaining string length

[payl_chdhry]

Hi,

I want to do masking for logs at index time but the replaced value ("X" here) should be same character length as original string. My requirement for masking is that for the value within [] only last 4 characters should be visible.

Example for below logs:

2020-08-18T13:17:43,990 [Engine 1] TRACE log data V01 [1|12345678]
2020-08-18T13:17:44,979 [Engine 2] TRACE log data V02 [2|A35453DFDF65]

The indexed logs should be:

2020-08-18T13:17:43,990 [Engine 1] TRACE log data V01 [1|XXXX5678]
2020-08-18T13:17:44,979 [Engine 2] TRACE log data V02 [2|XXXXXXXXDF65]

Currently I am using SEDCMD command as below but that is taking only static length for X:
s/(TRACE\slog\s+data\s+V\d+\s+\[\d\|)(\w+)(\w{4})/\1XXXXXX\3/g

Is there a way to replace string at index time with another string maintaining the count of  characters.

[thambisetty]

Is Length of the string dynamic? 

 

[payl_chdhry]

@thambisetty  yes the length would be dynamic. It should be anywhere between say 8 to maybe 20.

[thambisetty]

No worries 😉, I shared tested regex.

upvote if that solves your problem.

[cpetterborg · Accepted solution]

Using the clever regex from @thambisetty (and slightly modified), here is a SEDCMD that I have tested which works and does what you want:

SEDCMD=s/(?=[^\|]+\w{4}]$)./#/g

 

[payl_chdhry]

@cpetterborg Thank you for this. It worked 🙂

I had one more query in case you might be able to help. I have same requirement but for different  format of string. I am trying to customize this rex/sed for this format as well but not able to achieve it yet.

2020-08-19T07:42:38,942 [Engine 9] TRACE MEHSegment WHERE "00" "00000123456                 " 1 240

should give me:

2020-08-19T07:42:38,942 [Engine 9] TRACE MEHSegment WHERE "00" "0000012XXXX                 " 1 240

[thambisetty]

Can you post this as new question?

[payl_chdhry]

just did 🙂
https://community.splunk.com/t5/Getting-Data-In/Index-time-masking-maintaining-string-length-continu...

[thambisetty]

@payl_chdhry 
you could upvote my post also, because my post is the source to your answer.😉

[payl_chdhry]

Sorry m new to posting here. How do I upvote a post once I have marked anther post as answer.

[payl_chdhry]

great. Let me try this. @cpetterborg 

[to4kawa]

SEDCMD-mask = s/\|(\w+)(\w{4}\])$/|#\2/

Why can't we use this one, which has less of a load?

[payl_chdhry]

@to4kawa that would replace the whole  matching string with single #, like below:

2020-08-19T07:28:32,032 [Engine 8] TRACE SegmentLocks V13 added [1|A454FFDPP01]

To:

2020-08-19T07:28:32,032 [Engine 8] TRACE SegmentLocks V13 added [1|#PP01]

My requirement is to replace each character (and not whole string) with X so the count of characters replaced before masking is equal to count of Xs.

2020-08-19T07:28:32,032 [Engine 8] TRACE SegmentLocks V13 added [1|XXXXXXXPP01]