I'm working on a streaming custom command that converts a field containing binary to a multivalue field of the binary bit values, but whenever an existing multivalue field is passed in to my command it seems to get converted to a space separated string.
| append [| stats count as dummy] | eval dummymv="test1 test2 test3" | makemv dummymv delim=" " | eval test1="1100101" | eval test2=test1
And after running it through my command...
| append [| stats count as dummy] | eval dummymv="test1 test2 test3" | makemv dummymv delim=" " | eval test1="1100101" | eval test2=test1 | frombitmask field=test1
My command:
def stream(self, events):
for event in events:
#skip the event if it doesn't contain the field, or the field isn't binary
if self.field not in event.keys() or re.match("^(0b)?[0-1]+$", event[self.field], re.I) is None:
yield event
continue
try:
initial = event[self.field];
event[self.field] = []
for i, c in enumerate(reversed(initial)):
if c == "0" or c == "1":
value = int(c, 2)*((2**i))
if value > 0:
event[self.field].append(str(value))
else:
break
except Exception, e:
if not self.suppress_error:
raise e
yield event
commands.conf
[frombitmask]
type = python
filename = frombitmask.py
streaming = true
supports_getinfo = true
supports_multivalues = true
outputheader = true
I've tried the sample streaming command (countmatches) in the python SDK and it seems to suffer from this issue as well.
I also read through the SDK documentation on searchcommands, and while point #8 mentions fields using the naming convention "__mv_somefieldname" for multivalue fields, the events dictionary (in my command above) never contains any fields named like this.
How can I prevent multivalue fields from losing their multivaluedness in a streaming custom command?
Update
Further testing reveals that, while the __GETINFO__ call is indicating to Splunk that multivalues are supported:
stdout
changes_colorder,clear_required_fields,enableheader,generating,local,maxinputs,needs_empty_results,outputheader,overrides_timeorder,passauth,perf_warn_limit,required_fields,requires_srinfo,retainsevents,run_in_preview,stderr_dest,streaming,supports_multivalues,supports_rawargs,__mv_changes_colorder,__mv_clear_required_fields,__mv_enableheader,__mv_generating,__mv_local,__mv_maxinputs,__mv_needs_empty_results,__mv_outputheader,__mv_overrides_timeorder,__mv_passauth,__mv_perf_warn_limit,__mv_required_fields,__mv_requires_srinfo,__mv_retainsevents,__mv_run_in_preview,__mv_stderr_dest,__mv_streaming,__mv_supports_multivalues,__mv_supports_rawargs
1,0,1,0,0,0,1,1,0,0,0,,0,1,1,log,1,1,1,,,,,,,,,,,,,,,,,,,
Splunk is not sending any __mv_ fields when passing data to the custom command:
stdin
dummy,dummymv,test1,test2
0,"test1 test2 test3",1100101,1100101
I'm running Splunk 6.2.0, is this perhaps a known issue?
