Splunk Search

rex search returning everything instead of the one I'm trying to search, what am I doing wrong?

Jeet
Explorer

My rex search is returning all the rows instead of the one being searched. What am I doing wrong?

index=cloudwatchlogs loggroup="/aws-glue/jobs/xxxxx/*" meta_region="us-east-1" meta_env="TEST" meta_type="aws:jobs" | rex field="message.message" max_match=0 "Total rows from Raw Call meta:\s(?<msg1>\d+)\s" | rex field="message.message" max_match=0 "Total Meta rows written to S3 bucket:\s(?<msg2>\d+)\s" | rex field="message.message" max_match=0 "Total QCI Raw Data rows read from S3 bucket:\s(?<msg3>\d+)\s" | rex field="message.message" max_match=0 "Total root rows written to S3 bucket:\s(?<msg4>\d+)\s"

Sample data -

INFO:__main__:Total rows from Raw Call meta: 3995
INFO:__main__:Deleting duplicate rows
INFO:__main__:Total rows before Deleting duplicate rows: 3995
INFO:__main__:Listing duplicates, if any
INFO:__main__:Total Meta rows written to S3 bucket: 3995
INFO:__main__:Processing RAW QCI Data.

Labels (1)
0 Karma
1 Solution

andrew_nelson
Communicator

rex isnt a search filter function, its only for field extraction or data replacement.
See: rex - Splunk Documentation

 

In your search, your filter criteria is: 

index=cloudwatchlogs loggroup="/aws-glue/jobs/xxxxx/*" meta_region="us-east-1" meta_env="TEST" meta_type="aws:jobs"

 

Is the sample data one event with 6 lines or 6 separate events ? 

View solution in original post

0 Karma

andrew_nelson
Communicator

rex isnt a search filter function, its only for field extraction or data replacement.
See: rex - Splunk Documentation

 

In your search, your filter criteria is: 

index=cloudwatchlogs loggroup="/aws-glue/jobs/xxxxx/*" meta_region="us-east-1" meta_env="TEST" meta_type="aws:jobs"

 

Is the sample data one event with 6 lines or 6 separate events ? 

0 Karma

Jeet
Explorer

Got it. It's one event with 6 lines.

Tags (1)
0 Karma

andrew_nelson
Communicator

Ok. Have a look into using rex mode=sed in the documentation above and you should be able to replace any lines that don't end in a number with nothing. 

Something like this: 

| rex field=_raw mode=sed "s/[\n\r]+.*\D([\n\r]+|$)|$/\n/g"

 

Checks for any line that doesn't end number and replaces it with a new line character.

 

 

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...