Splunk Search

Find and filter out entries from log

tp92222
Explorer

I want to process log file which contains numeric entries like
eg

232231XX
232313XX
332133XX
411111XX
522222XX
734444XX
345555XX

i want to filter out entries in which digit is getting repeated more then 2 times continuously after 1 st digit

for eg from above entries
411111XX
522222XX
734444XX
345555XX

Thank you

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

View solution in original post

0 Karma

sundareshr
Legend

Try this

| rex field=yourfield "(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)

Now, if 77734444XX should NOT be filtered out, because it repeats only twice after the first digit, use this

| rex field=yourfield "(?<repeatednum>\d(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)
0 Karma

inventsekar
SplunkTrust
SplunkTrust

for eg from above entries
411111XX
522222XX
734444XX --- after the first digit 7, digit 3 is not repeating two times.
345555XX --- after the first digit 3, digit 4 is not repeating two times.

0 Karma

tp92222
Explorer

after 1 digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)
0 Karma

tp92222
Explorer

Thanx for quick reply .but i am looking for little different out put
after 1st digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

The updated answer should take care of that.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...