Splunk Search

rex - multiple matches and using |

jshaynes
Explorer

I'm trying to run several field extractions using the rex command. Here is a sample log format:

ironportmail: Info: MID 42342 ICID 1234 From: xyz@yyx.com
ironportmail: Info: MID 42342 ICID 1234 To: abc@def.com

I'd like to extract the MID, ICID, From and To fields using just one rex command. (This is so I can wrap an eventtype around it.)

Conceptually I'm looking for something like this, but havent been able to get it to work:

rex field=_raw "MID (?[^ ]+) | ICID (?[^ ]+) | From: <(?[^>]+)> | To: <(?[^>]+)"

Tags (1)
0 Karma
1 Solution

Ayn
Legend

I think you'd be better off putting this in props.conf / transforms.conf instead so you won't have to specify your field extractions explicitly in each search that you want to use these fields.

That said, this will work, though with a caveat:

... | rex field=_raw max_match=10 "(?:MID (?<mid>[^ ]+)|ICID (?<icid>[^ ]+)|From: <(?<mailfrom>[^>]+)>|To: <(?<mailto>[^>]+))"

The caveat being that you will need to provide rex with a max_match (default is 1), otherwise it will stop once it finds its first match.

View solution in original post

Ayn
Legend

I think you'd be better off putting this in props.conf / transforms.conf instead so you won't have to specify your field extractions explicitly in each search that you want to use these fields.

That said, this will work, though with a caveat:

... | rex field=_raw max_match=10 "(?:MID (?<mid>[^ ]+)|ICID (?<icid>[^ ]+)|From: <(?<mailfrom>[^>]+)>|To: <(?<mailto>[^>]+))"

The caveat being that you will need to provide rex with a max_match (default is 1), otherwise it will stop once it finds its first match.

jshaynes
Explorer

couple of follow up questions:

  1. what does the ?: at the beginning do. the query seems to work fine without it
  2. any way to prevent all of the empty matches that get returned as a result of changing max_match?

ex, I get a bunch of empty mailfrom, mailto, etc. fields since they don't all occur on every line:

sample field list from search results:
mailfrom="" | mailfrom="" | mailfrom="" | mailfrom=""abc@bvd.com" | mailto="" | mailto="" | mailto="xyz@abc.com" | ... etc

0 Karma

jshaynes
Explorer

awesome, that did it. thanks.

the reason im running these inline is because I am searching off of a summary index. those fields do already get extracted automatically for regular searches, but since the fields get dropped after doing a " | collect index=summary", im having to recreate them.

0 Karma
Get Updates on the Splunk Community!

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...

Admin Console: A Single, Unified Interface for All Your Cloud Admin Needs

WATCH NOWJoin us to learn how the admin console can save you time and give you more control over the Splunk® ...