I'm trying to write a new custom search command, more specifically a reporting command. I'm using the Python SDK 1.6.2.
I took the searchcommand template report.py. I removed the map function, as I want to do only a reduce operation.
If I run my custom reporting command in a search after a stats
command, it works pretty well.
If I run it as the first reporting command, I get the following error:
External search command 'toxlsx' returned error code 1. Script output = "error_message=AttributeError at "/opt/splunk/etc/apps/ta-toxlsx/bin/splunklib/searchcommands/reporting_command.py", line 88 : 'function' object has no attribute 'ConfigurationSettings' "
I have the following decorator for the reporting command class:
@Configuration(run_in_preview=False)
class toxlsxCommand(ReportingCommand):
and nothing for the reduce:
@Configuration()
def reduce(self, events):
Any idea what's going on here?
So, I added a map function that returns all records:
@Configuration()
def map(self, records):
return records
It seems to work (no errors, my script receives all data and works as expected), but I have absolutely no idea what I'm doing.
Is that the correct way to not do anything in the map phase? (especially, what is the difference with yield() vs what I'm doing here?)
Hi grundsch,
Would it be possible if you shared you code please?
I've been stuck on creating a reporting command for a few weeks now. I'm getting the following error even though I've tried doing what you said above:
Error: “ERROR dispatchRunner - RunDispatch::runDispatchThread threw error: Search Factory: Unknown search command 'my'.”
Any help would be much appreciated.
Thanks,
Richie.
Hi Richie,
looks like you have another problem. Did you configure the command in the /local/commands.conf correctly?
i.e. something like:
[toxlsx]
filename = toxlsx.py
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
run_in_preview = false
(here my command is named toxlsx, and the file name and class name muss match).
You need to restart Splunk to let it know about the new command.
If it still doesn't work, try to make the example commands in the SDK, without changes.
Good luck,
Stéphane
My commands.conf file consists of the following:
[mycommand]
chunked=true
filename=mycommand.py
With these settings, the streaming command works just fine so I am assuming that a reporting command should work fine once I call the ReportingCommand class and appropriate methods in my own script.
Anyway, thanks for your reply. I'll keep playing around with it.
All the best,
Richie.
mmmh... if your command is named "mycommand", then the class must be
@Configuration() class
mycommandCommand(ReportingCommand):
Weird, when I use the StreamingCommand class, I do not need to append "Command" to my class name.
It works now since I have changed my class name from "MyCommand" to "MyCommandCommand".
Thank you for pointing that out Stéphane.
Regards,
Richie.
yep, this whole SDK could get more documentation. For the moment, I'm staying with my previous statement:
It seems to work, but I have
absolutely no idea what I'm doing.
😄
Yeah, I agree. its quite difficult to find clear instructions and examples for each search command type.
I found this presentation after some time of Googling. Maybe it will help shed some light in areas where you are uncertain.
https://conf.splunk.com/files/2017/slides/extending-spl-with-custom-search-commands-and-the-splunk-s...
Also, if, like me, you run into an issue where the base search is sending a lot of events to the script (over 150k) then try processing the events in the "map" function rather than the "reduce" function.
cool, thanks!
It looks like the map function is missing a decorator, causing the AttributeError provided in the question:
@Configuration()
def map(self, records):
You're doing the right thing with the class decorator, in specifying requires_preop=False
, but be sure to return NotImplemented
in the map function.
Although it uses a map function, looking at the example 'sum' reporting command may also help: https://github.com/splunk/splunk-sdk-python/blob/master/examples/searchcommands_app/package/bin/sum....
I added a map function, but it leads to the same error:
External search command 'toxlsx' returned error code 1. Script output = "error_message=TypeError at "/opt/splunk-6.5.2/etc/apps/ta-toxls/bin/splunklib/searchcommands/internals.py", line 519 : 'NotImplementedType' object is not iterable "
@Configuration()
def map(self, records):
return NotImplemented
You removed the map function, but did you remove the if phase=='map':
conditional statement in the prepare function?
def map(self, records):
""" Override this method to compute partial results.
:param records:
:type records:
You must override this method, if :code:`requires_preop=True`.
"""
return NotImplemented
def prepare(self):
phase = self.phase
if phase == 'map':
# noinspection PyUnresolvedReferences
self._configuration = self.map.ConfigurationSettings(self)
return
if phase == 'reduce':
streaming_preop = chain((self.name, 'phase="map"', str(self._options)), self.fieldnames)
self._configuration.streaming_preop = ' '.join(streaming_preop)
return
raise RuntimeError('Unrecognized reporting command phase: {}'.format(json_encode_string(unicode(phase))))
This would explain why it works after a stats (phase=reduce at that point) and not before a stats because it would still possibly be in a mapping phase at that time.
No, I haven't removed the condition in the prepare fuction...
The (reporting command) template show nothing about it:
#!/usr/bin/env python
import sys
from splunklib.searchcommands import \
dispatch, ReportingCommand, Configuration, Option, validators
@Configuration()
class %(command.title())Command(ReportingCommand):
""" %(synopsis)
##Syntax
%(syntax)
##Description
%(description)
"""
@Configuration()
def map(self, events):
# Put your streaming preop implementation here, or remove the map method,
# if you have no need for a streaming preop
pass
def reduce(self, events):
# Put your reporting implementation
pass
dispatch(%(command.title())Command, sys.argv, sys.stdin, sys.stdout, __name__)
and the prepare function is only found in the ReportingCommand class (splunklib/searchcommands/reporting_command.py) without info about overriding the prepare function...
I tried to remove the map section completely, or keep the return only, but it doesn't help. It triggers other errors:
with only return:
/splunklib/searchcommands/internals.py", line 519 : 'NotImplementedType' object is not iterable
(which shows that it tries to use the map() function of the base class, which returns "NotImplemented")
It triggers the exception, as the phase is not recognised.
So basically, I think I need to implement a dummy map function that does nothing. Any idea how this would look like?
any chance to see a bit more code for reporting_command.py?