Extracting fields from a multi line log, with mulitple itterations of data

jamesdon — Mon, 28 Sep 2020 09:32:41 GMT

Hello,

I am extracting logs from the results of a screen scrape on Cisco load balancers. I used to use some Perl code to pre-parse the data, but I am trying to port that functionality into Splunk. Below is a some same output:

slb/prod16# show service-policy LB_123_ABC_POLICY detail

Status : ACTIVE

Description: NONE

Interface: vlan 1180 4713 6404

service-policy: LB_VIP_POLICY

class: L4_SSL-123_ABC_CLASS_DEV

  ssl-proxy server: NEW-SSL-PROXY

 VIP Address:    Protocol:  Port:

 10.47.15.16 tcp        eq    443  

  loadbalance:

    L7 loadbalance policy: L7_SSL-123_ABC_POLICY

    VIP Route Metric     : 77

    VIP Route Advertise  : DISABLED

    VIP ICMP Reply       : ENABLED-WHEN-ACTIVE

    VIP State: INSERVICE

    curr conns       : 780         , hit count        : 1251767   

    dropped conns    : 159774    

    client pkt count : 21301490  , client byte count: 2624639829          

    server pkt count : 26552109  , server byte count: 28101998657         

    conn-rate-limit      : 0         , drop-count : 0         

    bandwidth-rate-limit : 0         , drop-count : 0         

    L7 Loadbalance policy : L7_SSL-123_ABC_POLICY

      class/match : class-default

        ssl-proxy client : SSL_PSRVICE_CLIENT

        LB action: : 

           primary serverfarm: prod.thingy:30011

                state: UP

              backup serverfarm : -

        hit count        : 1244324   

        dropped conns    : 1         

class: prod.thingy:30010

 VIP Address:    Protocol:  Port:

 10.47.15.16 tcp        eq    80   

  loadbalance:

    L7 loadbalance policy: prod.thingy:8010

    VIP Route Metric     : 77

    VIP Route Advertise  : DISABLED

    VIP ICMP Reply       : ENABLED

    VIP State: INSERVICE

    curr conns       : 602         , hit count        : 308787    

    dropped conns    : 35        

    client pkt count : 1526460   , client byte count: 130340865           

    server pkt count : 1224022   , server byte count: 208605432           

    conn-rate-limit      : 0         , drop-count : 0         

    bandwidth-rate-limit : 0         , drop-count : 0         

    L7 Loadbalance policy : prod.thingy:8010

      class/match : class-default

        LB action: : 

           primary serverfarm: prod.thingy:30010

                state: UP

              backup serverfarm : -

        hit count        : 308784    

        dropped conns    : 0

This is the detail of two Classes, but IRL, there are many more Classes listed one after another. I am trying to map the "class:" name with the corresponding "curr conns" that follows.

Is this possible? I am thinking that I would be able to grab the other data points if it is ever requested in the future. Would I be better off pre-parsing the data, like I have in the past?

Thank you,

Jim

Re: Extracting fields from a multi line log, with mulitple itterations of data

Simeon — Mon, 28 Sep 2020 09:32:46 GMT

Sounds like you should either:

preparse the data and separate into individual classes
create a multivalue field that takes the initial class value as input for the field name of curr_conns (must be an indexed field to do this)

You can probably get away with doing some fancy regexes, but you will need to use the multivalue field parameters (MV_ADD in config files, max_matches when using rex).

Re: Extracting fields from a multi line log, with mulitple itterations of data

jamesdon — Wed, 18 May 2011 16:53:46 GMT

OK, I will keep it simple and pre-parse.

topic Re: Extracting fields from a multi line log, with mulitple itterations of data in Getting Data In

Extracting fields from a multi line log, with mulitple itterations of data

Re: Extracting fields from a multi line log, with mulitple itterations of data

Re: Extracting fields from a multi line log, with mulitple itterations of data