Splunk Search

Find and filter out entries from log

tp92222
Explorer

I want to process log file which contains numeric entries like
eg

232231XX
232313XX
332133XX
411111XX
522222XX
734444XX
345555XX

i want to filter out entries in which digit is getting repeated more then 2 times continuously after 1 st digit

for eg from above entries
411111XX
522222XX
734444XX
345555XX

Thank you

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

View solution in original post

0 Karma

sundareshr
Legend

Try this

| rex field=yourfield "(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)

Now, if 77734444XX should NOT be filtered out, because it repeats only twice after the first digit, use this

| rex field=yourfield "(?<repeatednum>\d(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)
0 Karma

inventsekar
Super Champion

for eg from above entries
411111XX
522222XX
734444XX --- after the first digit 7, digit 3 is not repeating two times.
345555XX --- after the first digit 3, digit 4 is not repeating two times.

0 Karma

tp92222
Explorer

after 1 digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

View solution in original post

0 Karma

tp92222
Explorer

Thanx for quick reply .but i am looking for little different out put
after 1st digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

The updated answer should take care of that.

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!