Hi, novice splunker here.
How could I search or extract all the unique numbers while keeping certain digits masked?
E.g. Field with a 16 digits credit card number would look like 123456######0011.
I typed in this syntax :
source="log.xml" | dedup CC | rex field=CC mode=sed "s/(\\d{6})(\\d{6})(\\d{4})/\\1############\\3/g"
to no avail.
Your code works just fine. Here's some randomly generated 16-digit numbers starting with 4 or 5.
| makeresults
| eval CC = "4272244909012766 4558001333691191 4558001234691191 4558005678691191 4251094150032295 4389981912828463 5967770933229475 5301338379674613 5985771287844680 5006543747713681"
| makemv CC
| mvexpand CC
| rex field=CC mode=sed "s/(\\d{6})(\\d{6})(\\d{4})/\\1############\\3/g"
Note that since you are masking some of the characters, you may get the same output CC numbers for different inputs. Records 2, 3 and 4 produce the same masked number....
427224############2766
455800############1191
455800############1191
455800############1191
425109############2295
438998############8463
596777############9475
530133############4613
598577############4680
500654############3681
Note - it works just fine in search without doubling the backslashes also.
Why is this a new question? This clutters the forum (when you ask the same question twice).
Your code works just fine. Here's some randomly generated 16-digit numbers starting with 4 or 5.
| makeresults
| eval CC = "4272244909012766 4558001333691191 4558001234691191 4558005678691191 4251094150032295 4389981912828463 5967770933229475 5301338379674613 5985771287844680 5006543747713681"
| makemv CC
| mvexpand CC
| rex field=CC mode=sed "s/(\\d{6})(\\d{6})(\\d{4})/\\1############\\3/g"
Note that since you are masking some of the characters, you may get the same output CC numbers for different inputs. Records 2, 3 and 4 produce the same masked number....
427224############2766
455800############1191
455800############1191
455800############1191
425109############2295
438998############8463
596777############9475
530133############4613
598577############4680
500654############3681
Note - it works just fine in search without doubling the backslashes also.
What if the value being replaced was from DNS Query logs like:
(3)windows(12)live(2)com(7)
and sometimes
(4)live(7)windows(4)update(13)com(4)
The goal of course to replace (\d+) with a "." anywhere it appears in the field. I can easily replace the leading and trailing (\d) using rex to strip them from the field so it looks like:
windows(12)live(2)com
or
live(7)windows(4)update(13)com
how can I then use mode=sed to convert it to windows.live.com and/or live.windows.update.com? Sometimes the domain name is 10+ word.word.word.owrd in length and it is variable.
I figured it out for those curious:
| rex field=domain mode=sed "s/\(\d+\)/./g"
The following works:
| makeresults | eval raw="1234567890123456" | rex field=raw mode=sed "s/(\d{6})(\d{6})(\d{4})/\\1#######\\3/g"
So it is likely that the only thing wrong is the double backslashes.