Splunk Search

Custom Search Command - Can I emit multiple records for each input record?

kmarx
Explorer

I'm have a custom command that parses an input field in each given record and emits 0 to N records as its output. I'm doing this to avoid a bunch of mvzip/mvexp and logic in the calling SPL. It does seem to work fine, but I'd like some reasurance that this is really supported in the SDK.

The SDK doc for StreamingCommand of the PythonSDK (http://docs.splunk.com/Documentation/PythonSDK) says (bold highlighting mine),

Streaming commands typically filter, augment, or update, search result records. Splunk will send them in batches of up to 50,000 records....

This (and the rest of the article and others like it that I've found) don't really seem to specify how many records can be returned. The above seems to suggest that it's really supposed to be 1:1 and not 1:n.

Here's an edited down version of my code. (I'm also a Python newbie, so apologies for any ugliness there.)

import sys
from mytokeninfo import Info
sys.path.append("splunk_sdk-1.6.5-py2.7.egg")
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration

@Configuration(local=True)    # Per doc on "stateful" streaming commands
class ExStatefulCommand(StreamingCommand):
    def stream(self, records):
    for record in records:
         tokens = self.parseRecordForTokens(record)
         for token in tokens:
             info = self.processToken(token)
             record['newField1']  = info.field1  # Application specifics simplified here for clarity (hopefully)
             record['newField2'] = info.field2   # ...etc
             yield record

So, for each record, I'm augmenting it one or more times and also yield'ing it each time. If so, I'd love to see the doc for it. If not, can I get an explanation as to why and also suggestions for how best to deal with this in a clean and proper manner?

0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Shape the Future of Splunk: Join the Product Research Lab!

Join the Splunk Product Research Lab and connect with us in the Slack channel #product-research-lab to get ...

Auto-Injector for Everything Else: Making OpenTelemetry Truly Universal

You might have seen Splunk’s recent announcement about donating the OpenTelemetry Injector to the ...