We are developing a custom search command to create events, this is using a streaming command with version 2 of the protocol, as the source is quite slow we'd like to send smaller chunks of results back to Splunk than the default 50,000, e.g. chunks of 1,000 events, so that users can view the partial results sooner. We've tried various approaches including an incrementing integar and calling self.flush() when it is divisable by 1,000, but that caused a buffer full error. Any suggestions would be really appreciated     ... @Configuration(type='streaming') class OurSearchCommand(GeneratingCommand): ... for item in OurGenerator(): item['_time'] = item['timestamp'] yield item     ... View more

Hi, I'm new to creating custom search commands, and haven't been able to understand the examples in the SDK repo on GitHub, if someone can point me to or provide a step by step guide to achieve the below that'd be really appreciated. Problem: I need to filter my results to only include events where a field (cidr) is a subnet of a range entered on a dashboard ($search_cidr$). As the builtin cidrmatch(CIDR, IP) only works with an IP and a CIDR range, I don't beleive I can do this natively, so I'm thinking a custom where command is the way to go, so I can do something like this in SPL: | from datamodel:mymodel | where SubnetOf(cidr, "$search_cidr$") I've written a basic Python function in my apps bin directory to do the comparison, it takes either (IP, Subnet) or (Subnet. Subnet) and returns a boolean, however I don't know how to turn this into a filter for the where command to use.   #!/usr/bin/env python3 from ipaddress import ip_network def SubnetOf(Needle, Haystack): try: Needle = ip_network(Needle) Haystack = ip_network(Haystack) return Needle.subnet_of(Haystack) except: return False     Any guidance to get me started would be great. ... View more

I cannot find this question being asked this way round, so hopefully its not a duplicate. I have a lookup CSV like this: ip,ip-info,timestamp 1.2.3.4,Text about the IP,2020-04-16T17:20:00 4.3.2.1,Different Text Here,2020-01-01T09:00:00 My log source summaries IPs to CIDR subnet, I have no control over this, extracted field looks like this cidr=1.2.3.0/24 . I need to use the CIDR from the log source to get the entrie(s) within that subnet via a lookup, e.g. index=myindex | lookup ip-info ip AS cidr OUTPUT ip-info timestamp AS ip-info-timestamp I have tried adding match_type = CIDR(ip) to the [ip-info] stanza in $SPLUNK_HOME/etc/apps/search/local/transforms.conf but I think this is for looking up an IP against CIDR masks, rather than the way round I need, as no results are returned. Is this possible, if so, how do I achieve it? Thanks ... View more

I am trying to test Splunk DB Connect (on a DEVTEST instance CentOS 8, Splunk 8.0.2, Splunk DB Connect 3.3.0, openjdk 11.0.6, mysql-connector-java-5.1.48-bin.jar) to connect to a remote MariaDB 5.5.5 server. I think I followed the install instructions correctly; except I cannot do the "Check DB Connect installation health" steps as Health Check doesn't seem to be available on my DEVTEST instance (Browser Console shows: GET http://splunk:8000/en-US/splunkd/__raw/services/search/distributed/groups?output_mode=json&_=1585579039878 402 (Payment Required) .) Using SQL Explorer in Splunk DB Connect, I can select my Collection, Catalog and Table from the drop downs on the left, which auto-creates a SELECT statement, the connection must be up and properly authenticated to populate the Catalog and Table dropdowns, but when I click Run no results are returned and the following is written to splunk_app_db_connect_dbxquery.2020-03-30.log : 2020-03-30 15:32:58.031 [main] INFO com.splunk.dbx.command.DbxQueryServer - operation= connection_name= stanza_name= action=dbxquery_server got request 2020-03-30 15:32:58.063 [main] INFO com.splunk.dbx.command.DbxQueryServer - operation= connection_name= stanza_name= action=dbxquery_server got request 2020-03-30 15:32:58.093 [main] INFO com.splunk.dbx.command.DbxQueryServer - operation= connection_name= stanza_name= action=dbxquery_server got request 2020-03-30 15:32:58.121 [main] INFO com.splunk.dbx.command.DbxQueryServer - operation= connection_name= stanza_name= action=dbxquery_server got request 2020-03-30 15:32:58.156 24727@splunk [DBX-QUERY-WORKER-60] ERROR com.splunk.dbx.command.DbxQueryCommand - operation= connection_name= stanza_name= action=dbxquery_command failed to get connection com.splunk.dbx.exception.NotFoundException: Can not find object MISPReader of type connection. at com.splunk.dbx.command.DbxQueryCommand.lambda$getConnection$1(DbxQueryCommand.java:180) at java.base/java.util.Optional.orElseThrow(Optional.java:408) at com.splunk.dbx.command.DbxQueryCommand.getConnection(DbxQueryCommand.java:180) at com.splunk.dbx.command.DbxQueryCommand.generate(DbxQueryCommand.java:359) at com.splunk.search.command.GeneratingCommand.process(GeneratingCommand.java:183) at com.splunk.search.command.ChunkedCommandDriver.execute(ChunkedCommandDriver.java:110) at com.splunk.search.command.AbstractSearchCommand.run(AbstractSearchCommand.java:50) at com.splunk.search.command.GeneratingCommand.run(GeneratingCommand.java:15) at com.splunk.dbx.command.DbxQueryCommand.runCommand(DbxQueryCommand.java:256) at com.splunk.dbx.command.DbxQueryServer.lambda$handleQuery$1(DbxQueryServer.java:144) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) I've installed MySQL client on the Splunk server and can use this to connect and run the same SELECT query successfully, so permissions on DB are correct. When I try to set dbxquery log level to debug the UI reports ('%s stanza in commands conf file is not valid because there must exist one and only one attribute with a value -DDBX_COMMAND_LOG_LEVEL=${LOG_LEVEL}', 'dbxquery') I've been search for a few hours trying to find an answer but haven't been able to, any suggestions welcome. Thanks Joe ... View more