Splunk Search

Multiple Value Field Extraction Help

jaoui
Path Finder

I am trying to come up with a Regex that will extract several field values from an event which can potentially have several occurrences of each field in a given event:

One of the Regexes that I am working on so far is something like this:

... | rex "Hex-STRING:\s(?<macstatus>([a-fA-F0-9][a-fA-F0-9]){1}?)"

But i can't figure out how to account for multiple instances of the variable


Sample events:

2010-10-20 08:39:21 switch.local [UDP: [192.168.1.6]:54862->[192.168.1.16]]: Trap , DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (977272346) 113 days, 2:38:43.46, SNMPv2-MIB::snmpTrapOID.0 = OID: CISCO-MAC-NOTIFICATION-MIB::cmnMacChangedNotification, CISCO-MAC-NOTIFICATION-MIB::cmnHistMacChangedMsg.24 = Hex-STRING: <font color=22345>02 01 0F D4 9A 20 ED C6 7E 00 30 02 00 3C 00 23 32 2D E4 C0 01 F8 02 03 06 00 23 32 2D E4 C0 01 F8 00</font> , CISCO-MAC-NOTIFICATION-MIB::cmnHistTimestamp.24 = Wrong Type (should be Timeticks): INTEGER: 977272346
2010-10-20 08:40:21 switch.local [UDP: [192.168.1.6]:62723->[192.168.1.16]]: Trap , DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (977278346) 113 days, 2:39:43.46, SNMPv2-MIB::snmpTrapOID.0 = OID: CISCO-MAC-NOTIFICATION-MIB::cmnMacChangedNotification, CISCO-MAC-NOTIFICATION-MIB::cmnHistMacChangedMsg.25 = Hex-STRING: 01 01 0F D4 9A 20 ED C6 7E 00 30 00 , CISCO-MAC-NOTIFICATION-MIB::cmnHistTimestamp.25 = Wrong Type (should be Timeticks): INTEGER: 977278346


The string of Hex characters contains the information I am looking for.
Every 11 octets of Hex represents one group.

For example, this would be one group (the final octet of '00' in each event marks the end of the string):

01 01 0F D4 9A 20 ED C6 7E 00 30

Each group consists of four fields.

  1. mac_status = Octet #1
    • Ex: (01)

  2. mac_vlan = Octet #2,3
    • Ex: (01 0F)

  3. mac_addr = Octet #4,5,6,7,8,9
    • Ex: (D4 9A 20 ED C6 7E)

  4. mac_portid = Octet #10,11
    • Ex: (00 30)

Thank you, David

1 Solution

Ledion_Bitincka
Splunk Employee
Splunk Employee

You can have multiple capturing groups in your regex. Is this what you're looking for?

... | rex "Hex-STRING: (?<mac_status>([a-fA-F0-9]{2}) (?<mac_vlan>[a-fA-F0-9]{2} [a-fA-F0-9]{2}) (?<mac_addr>[a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2}) (?<mac_portid>[a-fA-F0-9]{2} [a-fA-F0-9]{2}) 00"

View solution in original post

0 Karma

mikaelbje
Motivator

7 years after:

This one fixes an issue with the Hex STRING being split by a newline. Documenting this in case it is of value to someone.

| rex max_match=50000 field=_raw "(?msi)Hex-STRING:\s?(?<Hex_STRING>.+)\t"
| rex field=Hex_STRING mode=sed "s/\s\n/ /g"
| rex max_match=50000 field=Hex_STRING "(?<mac_status>([a-fA-F0-9][a-fA-F0-9]){1}) (?<mac_vlan>[a-fA-F0-9]{2} [a-fA-F0-9]{2}) (?<mac_addr>[a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2}) (?<mac_portid>[a-fA-F0-9]{2} [a-fA-F0-9]{2})"
| rex field=mac_vlan mode=sed "s/ //g"
| rex field=mac_portid mode=sed "s/ //g"
| rex field=mac_addr mode=sed "s/ /:/g"
| eval mac_addr=lower(mac_addr)
| stats latest(_time) AS _time values(host) AS host latest(mac_status) AS mac_status latest(mac_vlan) AS vlan latest(mac_portid) AS mac_portid BY mac_addr
| eval vlan=tonumber(vlan,16)
| eval mac_portid=tonumber(mac_portid,16)
| eval mac_status=tonumber(mac_status,16)
0 Karma

Ledion_Bitincka
Splunk Employee
Splunk Employee

You can have multiple capturing groups in your regex. Is this what you're looking for?

... | rex "Hex-STRING: (?<mac_status>([a-fA-F0-9]{2}) (?<mac_vlan>[a-fA-F0-9]{2} [a-fA-F0-9]{2}) (?<mac_addr>[a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2} [a-fA-F0-9]{2}) (?<mac_portid>[a-fA-F0-9]{2} [a-fA-F0-9]{2}) 00"
0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...