Extracting fields from SNMP GETBULK data

[adlireza]

Hi everyone,

I need to extract fields from data continuously polled for via SNMP Modular Input. Each event looks like this:

IF-MIB::ifInOctets."1" = "1587826952" IF-MIB::ifOutOctets."1" = "3472375195" IF-MIB::ifInOctets."2" = "0" IF-MIB::ifOutOctets."2" = "0" IF-MIB::ifInOctets."3" = "0" IF-MIB::ifOutOctets."3" = "0" IF-MIB::ifInOctets."4" = "0" IF-MIB::ifOutOctets."4" = "0" IF-MIB::ifInOctets."5" = "0" IF-MIB::ifOutOctets."5" = "0" IF-MIB::ifInOctets."6" = "50036733" IF-MIB::ifOutOctets."6" = "3575426650" IF-MIB::ifInOctets."7" = "0" IF-MIB::ifOutOctets."7" = "0" IF-MIB::ifInOctets."8" = "657176060" IF-MIB::ifOutOctets."8" = "2715199686"

I've tried using the extractions in snmp_ta but it didn't work for this particular set of SNMP poll data. It managed to create "ifInOctets" and "ifOutOctets" fields but does not separate them by interface ID.

I've then resorted to index-time extraction.

My transforms.conf:

[snmp_ifOctets_extraction]
REGEX=IF-MIB::(.+?)\.\"((?:\d\.?)+)\"\s=\s\"(.*?)\"
FORMAT=$1.$2::$3
WRITE_META = true
REPEAT_MATCH = true

My props.conf:

[snmp_ifoctets]
TRANSFORMS-ifoctets = snmp_ifOctets_extraction

That didn't turn out so well either. It extracted just one field, "ifInOctets.1". The regex didn't seem to repeat itself throughout the event even though REPEAT_MATCH is set. Anyone have any ideas why this is happening?

PS I'm also open to any other ideas on how to parse this set of data. I'm thinking of splitting each poll result into individual events next if index-time extraction isn't workable.

[jkat54 · Accepted solution]

Try adding this to the beginning of your regex your regex (?s)

 [snmp_ifOctets_extraction]
 REGEX=(?s)IF-MIB::(.+?)\.\"((?:\d\.?)+)\"\s=\s\"(.*?)\"
 FORMAT=$1.$2::$3
 WRITE_META = true
 REPEAT_MATCH = true

(?s) = dotall
(?m) = multiline
(?ms) = dotall, multiline

[adlireza]

Thanks. That will be my next course of action. I will be using the built-in option in the SNMP Modular Input app to split the output into individual lines.

[Muwafi]

Hello @adlireza

Hope you can update us with your working solution at the end.

Thanks

[jkat54]

Great, let us know how it works out. As is often the case, there are more than a few ways to "skin the cat".

[adlireza]

This doesn't work for me. It still only extracts one field.

I think the regex part should be OK. I've tested it against a sample of the results using RegExr and it is able to capture multiple groups there. I'm thinking it's something more to do with Splunk, maybe some setting in either transforms, props or fields.conf.

[jkat54]

Ok so is it worth it to break each of these fields into their own events using a line breaker regex?

I feel like the (?ms) might also work but maybe not since there's this repaet_match setting.

[jkat54]

And don't forget to disable should_linemerge if you go that route.