Well done on this triage! I have not looked into this issue in the last year or so, but I do see there is now a pull request on the issue in github, which I have not yet had a look at. https://github.com/splunk/splunk-sdk-python/issues/150 https://github.com/splunk/splunk-sdk-python/pull/251 ... View more

I downvoted this post because the other answer on this page is more correct ... View more

70000 is greater than the maximum of 10000 for a subsearch. You either need to transpose your results in the subsearch so that rows become columns, or move the large output to be the first query so it is no longer a subsearch. ... View more

By default, there is no field stored in the data that indicates which splunk instance that parsed/cooked the data. You only have splunk_server which is the indexer where the data for that search is on disk, and the host field which will be the source of the data, unless you have overwritten the host field with a transform, or explicitly set it to something else in inputs.conf. (And this is likely the case where you are receiving firewall logs) If this is really a requirement, you could create an additional indexed field (in the example below I call this splunk_parser) to store the data. On the heavy forwarder, in inputs.conf: [default] _meta = splunk_parser::<ENTERYOURHEAVYFORWARDERNAMEHERE> On your search head, configure fields.conf to let splunk know this is an indexed field: [splunk_parser] INDEXED = True INDEXED_VALUE = False ... View more

Your query is using join when it shouldn't the correct use for this is as I suggested. Note this is lookup, not inputlookup: index=myindex "searchterm" [| inputlookup "mylookup.csv" | fields src_ip] |stats values(field1) values(field2) by src_ip |lookup mylookup.csv src_ip ... View more

yes, this would show that. ... View more

Subsearches like you are calling with 'join' and 'append' are limited to 10000 rows by default. See http://docs.splunk.com/Documentation/Splunk/latest/Search/Aboutsubsearches#Subsearch_performance_considerations ... View more

The Splunk ML toollkit is sitting on top of a copy of Anaconda Python 2.7 sitting in Splunk_SA_Scientific_Python_linux_x86_64 (or other relevant OS/arch directory) It is possible to just download a copy of Anaconda 2.7 and then copy over the relevant modules. From my notes it looks like https://repo.anaconda.com/archive/Anaconda2-4.0.0-Linux-x86_64.sh out of https://repo.anaconda.com/archive/ is a good place to start. Install it on its own, and then install the relevant module, then copy the directory, and deps over to the splunk app module directory Splunk_SA_Scientific_Python_linux_x86_64/bin/linux_x86_64/lib/python2.7. When testing you need to make sure you execute via the splunk python to call the anaconda python as the install uses libraries from both. If you want to install them in a different Splunk app than the ML app, you will need to copy over and perhaps modify the exec_anaconda.py as per the instructions in the SA README. I have successfully done this to add pandas and pyarrow to a custom application which uses the base numpy package out of Splunk_SA_Scientific_Python_linux_x86_64. To use Splunk_SA_Scientific_Python, copy exec_anaconda.py into your app's bin directory. At the top of your custom search command, add the following preamble: #!/usr/bin/python import exec_anaconda exec_anaconda.exec_anaconda() # Put the rest of your imports below, e.g.: import numpy as np ... View more

The most perfomant option will be to look them up from the lookup after the stats command So: index=myindex "searchterm" [| inputlookup "mylookup.csv" | fields src_ip] | stats values(field1) values(field2) by src_ip | lookup mylookup.csv src_ip ... View more

I have just been looking at an issue to fetch data from an endpoint that requires this approach. What I found is that setting the r.cookies values does NOT work, but setting the cookie directly in the header does... e..g: This doesn't work: if cookies: r.cookies = cookies but this does: if cookies and "myauthcookiename" in cookies: r.headers['Cookie'] = "myauthcookiename=" + cookies["myauthcookiename"]) ... View more

Hi, if you set the CHECK_METHOD in a props.conf on the universal forwarder to modtime, then the splunkd.log will have a entry when the file is created. E.g: If /opt/splunkforwarder/etc/system/local/inputs.conf had an entry: [monitor:///data/test/*.trg] And there was a /opt/splunkforwarder/etc/system/local/props.conf with an entry: [source::/data/test/*.trg] CHECK_METHOD = modtime Then when a file is created the splunkd.log will have an entry liek the following: # date Thu Jul 5 15:19:53 AEST 2018 # touch /data/test/my_new_file.trg # grep trg /opt/splunkforwarder/var/log/splunk/splunkd.log 07-05-2018 15:20:08.415 +1000 INFO WatchedFile - Will use tracking rule=modtime for file='/data/test/my_new_file.trg'. If these splunkd logs are forwarded, you can search them in the index=_internal ... View more

As an update, I've seen the ctypes stuff is only used in the directory sync portion of the ldap3 library, so removing that bit means the ctypes library isn't needed, and the app can stay portable... Just comment out these 2 lines: --- ldap3/protocol/microsoft.py.orig 2018-07-04 09:02:54.502926488 +1000 +++ ldap3/protocol/microsoft.py 2018-07-03 20:58:38.366085375 +1000 @@ -23,7 +23,7 @@ # along with ldap3 in the COPYING and COPYING.LESSER files. # If not, see <http://www.gnu.org/licenses/>. -import ctypes +#import ctypes from pyasn1.type.namedtype import NamedTypes, NamedType from pyasn1.type.tag import Tag, tagClassApplication, tagFormatConstructed @@ -112,7 +112,7 @@ flags |= 0x80000000 # converts flags to signed 32 bit (AD expects a 4 bytes long unsigned integer, but ASN.1 Integer type is signed # so the BER encoder gives back a 5 bytes long signed integer - flags = ctypes.c_long(flags & 0xFFFFFFFF).value + #flags = ctypes.c_long(flags & 0xFFFFFFFF).value control_value.setComponentByName('Flags', flags) control_value.setComponentByName('MaxBytes', max_length) ... View more

Hi, can you send through the search from the search bar that made the graph in your original post? (If it is in a dashboard, when you mouseover it, you can select the magnifying lens icon to open the underlying search) ... View more

I have had good results by upgrading the version of ldap3 that ships with the package. See my answer in https://answers.splunk.com/answers/587284/poor-ldapsearch-performance-sa-ldapsearch-app.html?childToView=667314#answer-667314 ... View more