I had to write a script that solves the IP address from a field to the corresponding BGP AS Number.
At this point, my script is somehow working and i can use it in the Splunk search web UI like this :
index=my_index | head 1000 | iptoas asdataorigin=Ip | stats count by ASCountry
My problem is that in order to use my custom command with the stats command, I need to run the search in verbose mode. Using the smart or fast mode will throw an error because the field my scripts need is not available. My understanding is that when using the stats with fast/smart modes, the search process will only fetch the fields that are necessary to compute the stats (like the ASCountry in this case), which means my script is unable to get feeded with the needed fields, hence the failure. If I do stats on the field that my scripts needs (the IP field) it's not failing, but it's obviously not providing the stats I need since I want to make stats on AS numbers...
While I don't mind waiting a few more seconds when using the verbose mode, this problem actually means that I'm unable to use my custom command to create dashboards, since they require smart orfast mode.
My Splunk version is 6.3.2
My Splunk SDK version is 1.5.0
My script looks like this :
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators
srcfield = Option(doc=''' field to get IP value from ''',default='Ip')
asdstfield = Option(doc=''' dest field for as number ''',default='ASNumber')
asdataorigin = Option(doc=''' dest field for ias data origin (redis-cache or dns) ''',default=False)
descdstfield=Option(doc=''' dest field for as number ''',default="ASName")
countrydstfield=Option(doc=''' dest field for as number ''',default="ASCountry")
def stream(self, events):
for event in events:
# for readability reason i'm only keeping the structure
dispatch(iptoas, sys.argv, sys.stdin, sys.stdout, __name__)
My default/command.conf is the following :
# Configured for Search Command Protocol version 1 by default
# Replace the contents of this file with commands-scpv2.conf to enable Search Command Protocol version 2
filename = iptoas.py
maxinputs = 10000000
chunked = true
streaming = true
supports_multivalues = true
retainsevents = true
I tried to get inspiration from the "iplocation" command which is bundled with Splunk and that happens to work perfectly in fast|smart mode, but the command is not an external command anymore. I'm not sure what to do.
Do this means that I can't write custom functions with python that works with the fast mode ? Did I forget something in the command.conf file or somewhere else ? Or maybe is there a way to force the smart mode to fetch the fields I need ? (i mean, besides using the field in the stats command).
Any help, tip or trick would be enlightening.
Thanks for your help!
... View more