Splunk Search

Custom Search Command - Can I emit multiple records for each input record?

kmarx
Explorer

I'm have a custom command that parses an input field in each given record and emits 0 to N records as its output. I'm doing this to avoid a bunch of mvzip/mvexp and logic in the calling SPL. It does seem to work fine, but I'd like some reasurance that this is really supported in the SDK.

The SDK doc for StreamingCommand of the PythonSDK (http://docs.splunk.com/Documentation/PythonSDK) says (bold highlighting mine),

Streaming commands typically filter, augment, or update, search result records. Splunk will send them in batches of up to 50,000 records....

This (and the rest of the article and others like it that I've found) don't really seem to specify how many records can be returned. The above seems to suggest that it's really supposed to be 1:1 and not 1:n.

Here's an edited down version of my code. (I'm also a Python newbie, so apologies for any ugliness there.)

import sys
from mytokeninfo import Info
sys.path.append("splunk_sdk-1.6.5-py2.7.egg")
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration

@Configuration(local=True)    # Per doc on "stateful" streaming commands
class ExStatefulCommand(StreamingCommand):
    def stream(self, records):
    for record in records:
         tokens = self.parseRecordForTokens(record)
         for token in tokens:
             info = self.processToken(token)
             record['newField1']  = info.field1  # Application specifics simplified here for clarity (hopefully)
             record['newField2'] = info.field2   # ...etc
             yield record

So, for each record, I'm augmenting it one or more times and also yield'ing it each time. If so, I'd love to see the doc for it. If not, can I get an explanation as to why and also suggestions for how best to deal with this in a clean and proper manner?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...