Getting Data In

Index time masking maintaining string length

payl_chdhry
Explorer

Hi,

I want to do masking for logs at index time but the replaced value ("X" here) should be same character length as original string. My requirement for masking is that for the value within [] only last 4 characters should be visible.

Example for below logs:

2020-08-18T13:17:43,990 [Engine 1] TRACE log data V01 [1|12345678]
2020-08-18T13:17:44,979 [Engine 2] TRACE log data V02 [2|A35453DFDF65]

The indexed logs should be:

2020-08-18T13:17:43,990 [Engine 1] TRACE log data V01 [1|XXXX5678]
2020-08-18T13:17:44,979 [Engine 2] TRACE log data V02 [2|XXXXXXXXDF65]

 

Currently I am using SEDCMD command as below but that is taking only static length for X:
s/(TRACE\slog\s+data\s+V\d+\s+\[\d\|)(\w+)(\w{4})/\1XXXXXX\3/g

 

Is there a way to replace string at index time with another string maintaining the count of  characters.

Labels (4)
Tags (2)
0 Karma
1 Solution

cpetterborg
SplunkTrust
SplunkTrust

Using the clever regex from @thambisetty (and slightly modified), here is a SEDCMD that I have tested which works and does what you want:

SEDCMD=s/(?=[^\|]+\w{4}]$)./#/g

 Screen Shot 2020-08-18 at 6.26.53 PM.png

View solution in original post

thambisetty
Super Champion

Is Length of the string dynamic? 

33FA2DD2-35F3-456B-A0C3-E175B54A71E5.png

————————————
If this helps, give a like below.

payl_chdhry
Explorer

@thambisetty  yes the length would be dynamic. It should be anywhere between say 8 to maybe 20.

0 Karma

thambisetty
Super Champion

No worries 😉, I shared tested regex.

upvote if that solves your problem.

————————————
If this helps, give a like below.
0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Using the clever regex from @thambisetty (and slightly modified), here is a SEDCMD that I have tested which works and does what you want:

SEDCMD=s/(?=[^\|]+\w{4}]$)./#/g

 Screen Shot 2020-08-18 at 6.26.53 PM.png

View solution in original post

payl_chdhry
Explorer

@cpetterborg Thank you for this. It worked 🙂

I had one more query in case you might be able to help. I have same requirement but for different  format of string. I am trying to customize this rex/sed for this format as well but not able to achieve it yet.

 

2020-08-19T07:42:38,942 [Engine 9] TRACE MEHSegment WHERE "00" "00000123456                 " 1 240

should give me:

2020-08-19T07:42:38,942 [Engine 9] TRACE MEHSegment WHERE "00" "0000012XXXX                 " 1 240

0 Karma

thambisetty
Super Champion

Can you post this as new question?

————————————
If this helps, give a like below.
0 Karma

payl_chdhry
Explorer
0 Karma

thambisetty
Super Champion

@payl_chdhry 
you could upvote my post also, because my post is the source to your answer.😉

————————————
If this helps, give a like below.
0 Karma

payl_chdhry
Explorer

Sorry m new to posting here. How do I upvote a post once I have marked anther post as answer.

0 Karma

payl_chdhry
Explorer

great. Let me try this. @cpetterborg 

0 Karma

to4kawa
Ultra Champion

SEDCMD-mask = s/\|(\w+)(\w{4}\])$/|#\2/

Why can't we use this one, which has less of a load?

0 Karma

payl_chdhry
Explorer

@to4kawa that would replace the whole  matching string with single #, like below:

My requirement is to replace each character (and not whole string) with X so the count of characters replaced before masking is equal to count of Xs.

2020-08-19T07:28:32,032 [Engine 8] TRACE SegmentLocks V13 added [1|XXXXXXXPP01]

Tags (1)
0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!