Splunk Search

Matching two expressions to one field

markmcd
Path Finder

I am trying to extract a field from logs that look like this:

Apr 28 07:45:22.992 On [2:18]20.5.4.1:5070 sent to 102.11.130.135:50953 
...
Apr 28 07:45:22.992 On [0:51]10.20.33.50:5060 received from 10.20.1.1:59758 
...

The fields I'm trying to extract are source & destination IPs for each entry. So for source_ip, it's 20.5.4.1 and 10.2.1.1. For the destination, it's 102.11.130.135 & 10.20.33.50.

It looks like I need a regex that matches a field that 'begins with "sent to" OR ends with "received from"' and vice-versa but I can't for the life of me get the regex to work.

I tried to use prefixes but ended up with some nasty regexes that just don't work.

(?i)([^\]\n]*\]|received from )(?P<FIELDNAME>\d+\.\d+\.\d+\.\d+:\d+)

Can I do this with Splunk? Is it possible to use two regexes to extract to one field?

1 Solution

Ayn
Legend

Sure. You could either simply use two separate field extraction rules that both write their results to a field with the same name, or you could use one extraction regex that picks up both cases. I think the first approach is way better so I'll just cover that.

Simply create an extraction like you would otherwise (field extraction tool in Splunk web, extractions in the Manager, props.conf / transforms.conf...) and then create another one for your 2nd case, and use the same field name for them both. This is the simplest and in my opinion best approach, because you don't have to build an overly complex regex to cover for two different types of matches.

View solution in original post

Ayn
Legend

Sure. You could either simply use two separate field extraction rules that both write their results to a field with the same name, or you could use one extraction regex that picks up both cases. I think the first approach is way better so I'll just cover that.

Simply create an extraction like you would otherwise (field extraction tool in Splunk web, extractions in the Manager, props.conf / transforms.conf...) and then create another one for your 2nd case, and use the same field name for them both. This is the simplest and in my opinion best approach, because you don't have to build an overly complex regex to cover for two different types of matches.

ektasardana
Explorer

I tried to do the same as mentioned above but it says the field name already exists. How should I proceed?

0 Karma

Yorokobi
SplunkTrust
SplunkTrust

You cannot use a field name twice in a | rex statement:

index=_internal | rex "\d{3} (?<hostname>[^\ ]+) (?<hostname>[^\ ]+)"

To do something like that, you need two separate | rexes in your SPL:

index=_internal | rex "\d{3} (?<hostname>[^\ ]+)" | rex "\d{3} \w+ (?<hostname>[^\ ]+)"

To do this in props.conf (note the double field name extraction here :D) :

EXTRACT-order_no1   = Order (N|n)o: (?<order_no>(?<dw_order_no>\d+))
EXTRACT-order_no2   = <original-order-no>(?<order_no>(?<dw_order_no>\d+))
EXTRACT-order_no3   = <current-order-no>(?<order_no>(?<dw_order_no>\d+))
EXTRACT-order_no4   = ORDER_NO: (?<order_no>(?<dw_order_no>\d+))
EXTRACT-order_no5   = "ORDR":"(?<order_no>(?<dw_order_no>\d+))
EXTRACT-order_no6   = ORDR=(?<order_no>(?<dw_order_no>\d+))
EXTRACT-order_no7   = "orderNumber": "(?<order_no>(?<dw_order_no>\d+))
EXTRACT-order_no8   = order\/detail\/(?<order_no>(?<dw_order_no>\d+))

As with | rex you can only extract the name once per line but you can have many lines with to repeat the field name (?<field_name_here>...)

ektasardana
Explorer

This worked. Thank you:)

0 Karma

markmcd
Path Finder

For any future searchers, this needs to be in a conf file (e.g. etc/users/admin/search/local/props.conf) with the left-hand side of the '=' being unique, but the RHS using the same (?P.+)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...