topic Re: Extracting fields from SNMP GETBULK data in Splunk Search

Extracting fields from SNMP GETBULK data

adlireza — Thu, 31 Dec 2015 09:17:40 GMT

Hi everyone,

I need to extract fields from data continuously polled for via SNMP Modular Input. Each event looks like this:

IF-MIB::ifInOctets."1" = "1587826952" IF-MIB::ifOutOctets."1" = "3472375195" IF-MIB::ifInOctets."2" = "0" IF-MIB::ifOutOctets."2" = "0" IF-MIB::ifInOctets."3" = "0" IF-MIB::ifOutOctets."3" = "0" IF-MIB::ifInOctets."4" = "0" IF-MIB::ifOutOctets."4" = "0" IF-MIB::ifInOctets."5" = "0" IF-MIB::ifOutOctets."5" = "0" IF-MIB::ifInOctets."6" = "50036733" IF-MIB::ifOutOctets."6" = "3575426650" IF-MIB::ifInOctets."7" = "0" IF-MIB::ifOutOctets."7" = "0" IF-MIB::ifInOctets."8" = "657176060" IF-MIB::ifOutOctets."8" = "2715199686"

I've tried using the extractions in snmp_ta but it didn't work for this particular set of SNMP poll data. It managed to create "ifInOctets" and "ifOutOctets" fields but does not separate them by interface ID.

I've then resorted to index-time extraction.

My transforms.conf:

[snmp_ifOctets_extraction]
REGEX=IF-MIB::(.+?)\.\"((?:\d\.?)+)\"\s=\s\"(.*?)\"
FORMAT=$1.$2::$3
WRITE_META = true
REPEAT_MATCH = true

My props.conf:

[snmp_ifoctets]
TRANSFORMS-ifoctets = snmp_ifOctets_extraction

That didn't turn out so well either. It extracted just one field, "ifInOctets.1". The regex didn't seem to repeat itself throughout the event even though REPEAT_MATCH is set. Anyone have any ideas why this is happening?

PS I'm also open to any other ideas on how to parse this set of data. I'm thinking of splitting each poll result into individual events next if index-time extraction isn't workable.

Re: Extracting fields from SNMP GETBULK data

jkat54 — Fri, 01 Jan 2016 13:34:24 GMT

Try adding this to the beginning of your regex your regex (?s)

 [snmp_ifOctets_extraction]
 REGEX=(?s)IF-MIB::(.+?)\.\"((?:\d\.?)+)\"\s=\s\"(.*?)\"
 FORMAT=$1.$2::$3
 WRITE_META = true
 REPEAT_MATCH = true

(?s) = dotall
(?m) = multiline
(?ms) = dotall, multiline

Re: Extracting fields from SNMP GETBULK data

adlireza — Sat, 02 Jan 2016 06:42:55 GMT

This doesn't work for me. It still only extracts one field.

I think the regex part should be OK. I've tested it against a sample of the results using RegExr and it is able to capture multiple groups there. I'm thinking it's something more to do with Splunk, maybe some setting in either transforms, props or fields.conf.

Re: Extracting fields from SNMP GETBULK data

jkat54 — Sat, 02 Jan 2016 13:59:36 GMT

Ok so is it worth it to break each of these fields into their own events using a line breaker regex?

I feel like the (?ms) might also work but maybe not since there's this repaet_match setting.

Re: Extracting fields from SNMP GETBULK data

jkat54 — Sat, 02 Jan 2016 14:00:32 GMT

And don't forget to disable should_linemerge if you go that route.

Re: Extracting fields from SNMP GETBULK data

adlireza — Mon, 04 Jan 2016 03:36:50 GMT

Thanks. That will be my next course of action. I will be using the built-in option in the SNMP Modular Input app to split the output into individual lines.

Re: Extracting fields from SNMP GETBULK data

jkat54 — Mon, 04 Jan 2016 13:13:51 GMT

Great, let us know how it works out. As is often the case, there are more than a few ways to "skin the cat".

Re: Extracting fields from SNMP GETBULK data

Muwafi — Fri, 12 Apr 2019 00:29:02 GMT

Hello @adlireza

Hope you can update us with your working solution at the end.

Thanks