rex search returning everything instead of the one I'm trying to search, what am I doing wrong?

[Jeet]

My rex search is returning all the rows instead of the one being searched. What am I doing wrong?

index=cloudwatchlogs loggroup="/aws-glue/jobs/xxxxx/*" meta_region="us-east-1" meta_env="TEST" meta_type="aws:jobs" | rex field="message.message" max_match=0 "Total rows from Raw Call meta:\s(?<msg1>\d+)\s" | rex field="message.message" max_match=0 "Total Meta rows written to S3 bucket:\s(?<msg2>\d+)\s" | rex field="message.message" max_match=0 "Total QCI Raw Data rows read from S3 bucket:\s(?<msg3>\d+)\s" | rex field="message.message" max_match=0 "Total root rows written to S3 bucket:\s(?<msg4>\d+)\s"

Sample data -

INFO:__main__:Total rows from Raw Call meta: 3995
INFO:__main__:Deleting duplicate rows
INFO:__main__:Total rows before Deleting duplicate rows: 3995
INFO:__main__:Listing duplicates, if any
INFO:__main__:Total Meta rows written to S3 bucket: 3995
INFO:__main__:Processing RAW QCI Data.

[andrew_nelson · Accepted solution]

rex isnt a search filter function, its only for field extraction or data replacement.
See: rex - Splunk Documentation

In your search, your filter criteria is: 

index=cloudwatchlogs loggroup="/aws-glue/jobs/xxxxx/*" meta_region="us-east-1" meta_env="TEST" meta_type="aws:jobs"

Is the sample data one event with 6 lines or 6 separate events ? 

[Jeet]

Got it. It's one event with 6 lines.

[andrew_nelson]

Ok. Have a look into using rex mode=sed in the documentation above and you should be able to replace any lines that don't end in a number with nothing. 

Something like this: 

| rex field=_raw mode=sed "s/[\n\r]+.*\D([\n\r]+|$)|$/\n/g"

Checks for any line that doesn't end number and replaces it with a new line character.