Splunk Search

Find and filter out entries from log

tp92222
Explorer

I want to process log file which contains numeric entries like
eg

232231XX
232313XX
332133XX
411111XX
522222XX
734444XX
345555XX

i want to filter out entries in which digit is getting repeated more then 2 times continuously after 1 st digit

for eg from above entries
411111XX
522222XX
734444XX
345555XX

Thank you

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

View solution in original post

0 Karma

sundareshr
Legend

Try this

| rex field=yourfield "(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)

Now, if 77734444XX should NOT be filtered out, because it repeats only twice after the first digit, use this

| rex field=yourfield "(?<repeatednum>\d(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)
0 Karma

inventsekar
SplunkTrust
SplunkTrust

for eg from above entries
411111XX
522222XX
734444XX --- after the first digit 7, digit 3 is not repeating two times.
345555XX --- after the first digit 3, digit 4 is not repeating two times.

0 Karma

tp92222
Explorer

after 1 digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)
0 Karma

tp92222
Explorer

Thanx for quick reply .but i am looking for little different out put
after 1st digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

The updated answer should take care of that.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Unlocking Unified Insights: New Gigamon Federated Search App for Splunk

In today’s data-heavy environment, organizations are caught in a data distribution dilemma. As data volumes ...

GA: New Data Management App in Splunk Platform

Streamlining Data Management: Introducing a unified experience in Splunk Managing data at scale shouldn’t feel ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...