Developing for Splunk Enterprise

Multiple results using custom ReportingCommand

afaucher
New Member

Hello,

i'm trying to developed a custom ReportingCommand. Like the build-in command stats, I want only the global result on all my events and not the partial results from the reduce function being use multiples times.

I tried with the example given in the splunk sdk :

import os,sys

splunkhome = os.environ['SPLUNK_HOME']
sys.path.append(os.path.join(splunkhome, 'etc', 'apps', 'sum-dev', 'lib'))

from splunklib.searchcommands import dispatch, ReportingCommand, Configuration, Option, validators
from splunklib.searchcommands.validators import Fieldname
import splunk
import logging, logging.handlers


def setup_logging():
    logger = logging.getLogger('splunk.sumdev')
    SPLUNK_HOME = os.environ['SPLUNK_HOME']

    LOGGING_DEFAULT_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log.cfg')
    LOGGING_LOCAL_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log-local.cfg')
    LOGGING_STANZA_NAME = 'python'
    LOGGING_FILE_NAME = "sumdev.log"
    BASE_LOG_PATH = os.path.join('var', 'log', 'splunk')
    LOGGING_FORMAT = "%(asctime)s %(levelname)-s\t%(module)s:%(lineno)d - %(message)s"
    splunk_log_handler = logging.handlers.RotatingFileHandler(os.path.join(SPLUNK_HOME, BASE_LOG_PATH, LOGGING_FILE_NAME), mode='a')
    splunk_log_handler.setFormatter(logging.Formatter(LOGGING_FORMAT))
    logger.addHandler(splunk_log_handler)
    splunk.setupSplunkLogger(logger, LOGGING_DEFAULT_CONFIG_FILE, LOGGING_LOCAL_CONFIG_FILE, LOGGING_STANZA_NAME)
    return logger

@Configuration()
class SumCommand(ReportingCommand):

    total = Option(
        doc='''
        **Syntax:** **total=***<fieldname>*
        **Description:** Name of the field that will hold the computed sum''',
        require=True, validate=validators.Fieldname())

    @Configuration()
    def map(self, records):
        """ Computes sum(fieldname, 1, n) and stores the result in 'total' """
        self.logger.debug('SumCommand.map')
        fieldnames = self.fieldnames
        total = 0.0
        for record in records:
            for fieldname in fieldnames:
                total += float(record[fieldname])
        yield {self.total: total}

    @Configuration()
    def reduce(self, records):
        """ Computes sum(total, 1, N) and stores the result in 'total' """
        self.logger.debug('SumCommand.reduce')
        fieldname = self.total
        total = 0.0
        for record in records:
            value = record[fieldname]
            try:
                total += float(value)
            except ValueError:
                self.logger.debug('  could not convert %s value to float: %s', fieldname, repr(value))
        yield [{self.total: total}]


dispatch(SumCommand, sys.argv, sys.stdin, sys.stdout, __name__)

 

With that code, the search index=_internal | head 200 | sum total=lines linecount   gives me a field "lines" with multiples values, and not one value corresponding to the total count like I want to.

It's my first time doing a ReportingCommand, I will really appreciate anyone helps !


Labels (2)
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.