I suspect that may be an issue with Python compatibility.  Try forcing the App to use Python 2 by doing the following. Make a new directory called “local” in $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices. Copy the $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/default/inputs.conf and $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/default/restmap.conf files to the new $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/local directory. Edit the inputs.conf and restmap.conf files in the $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/local directory and change “python3” to “python2”. Restart Splunk. If that works, you can remove the changes in the future by deleting the $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/local directory when the compatibility issue is corrected. ... View more

I would make sure to search all inputs.conf file to make sure there are no hidden monitor stanzas anywhere. You can also use btool to do this. Then I would look at the ERROR and WARN messages in the $SPLUNK_HOME/var/log/splunk/splunkd.log file to see what useful information you can find there. If you still cannot determine the cause of the directory not emptying, I would file a support ticket to get additional assistance. I would suggest attaching a diag from the instance. Also provide as much information on when the behavior started, and what changed prior to noticing the issue. ... View more

That first sentence troubles me. The forwarders connect to Splunk Cloud via SSL. The configuration to make this happen is available in the Splunk_Forwarder_App as mentioned. The piece about allowing the forwarders to "talk to your Splunk Cloud instance over TCP port 9997" is not correct to the best of my knowledge. ... View more

As of more recent versions of Splunk (version 7.x and newer I believe) this works. ... View more

This 50 MB threshold is in older versions of Splunk. Starting with Splunk 6.6.0, the threshold value has been increased to 500 MB. concerningReplicatedFileSize = * Any individual file within a bundle that is larger than this value (in MB) will trigger a splunkd.log message. * Where possible, avoid replicating such files, e.g. by customizing your blacklists. * Defaults to: 500 ... View more

The automatic reaping of search artifacts occurs due to the passage of time. How much time is tracked in a variety of different ways via settings in the limits.conf and savedsearches.conf files. While working with the customer on this, we could see the requisite files in the search artifact were constantly being updated when the artifact was reaped. We asked if the dispatch directory was located on an NFS mount point, which it was. To test for a time skew issue, we created a file in the dispatch directory and checked the time of creation. We then ran the date command on the Splunk instance. The creation date of the test file was three minutes earlier than the date of the host computer which confirmed the time skew causing the searches to fail. The time on the NFS mount and the Splunk instance both need to be synchronized using NTP to prevent this type of issue from occurring. ... View more

The reason for the limit is to prevent excessive memory consumption. In the newer versions of Splunk, this value has been increased to 2GB. This is the information from the server.conf spec file. max_content_length = * Measured in bytes * HTTP requests over this size will rejected. * Exists to avoid allocating an unreasonable amount of memory from web requests * Defaulted to 2147483648 or 2GB * In environments where indexers have enormous amounts of RAM, this number can be reasonably increased to handle large quantities of bundle data. ... View more

Check to see if the app being deployed via the deployment server contains an install_source_checksum in its app.conf file. If it does, try removing that from the client and the deployment server and applying the change again. ... View more

You can view the XML View option from the Windows Event Viewer. Open an event in the Event Viewer Click on the "Details" tab Click on the "XML View" radio button This is the information which is ingested. ... View more

As jreuter stated, do not use maxHotSpanSec=86400. It has been proven this can cause undesired behavior if aligned with the hour or the day. ... View more

In order to send dashboards as PDF files, you need to have the admin_all_objects role assigned to the user. This is referenced in the documentation. User role configuration to schedule PDF delivery of dashboards For a user to schedule PDF delivery of dashboards, the user role must contain the following capabilities: • schedule_search • admin_all_objects This capability is required only if the mail host requires log-in credentials. ... View more

The Distributed Management Console (DMC) makes changes to a number of different files. One of which is the addition of search groups in distsearch.conf which will change the way in which searches behave. The configuration documentation contains the following warning. Important: Except for the case of a standalone, non-distributed Splunk Enterprise deployment, the instance hosting the DMC should not be used as a production search head and should not run any searches unrelated to its function as the DMC. Also, when enabling the distributed mode of the feature, the user is presented with the following screen. The only solution to correct this is to disable DMC on the search head. Instructions for doing this can be found in the documentation. ... View more

There was a bug (SPL-93913) in the early 6.2 versions of Splunk which prevented the sending of PDF reports in a search head cluster. This was fixed in version 6.2.2. ... View more

By default, Splunk uses TLSv1+HIGH:@STRENGTH. The ciphers used with this setting can be retrieved by running the following command. $SPLUNK_HOME/etc/splunk cmd openssl ciphers -v "TLSv1+HIGH:@STRENGTH" The ciphers used to communicate with the Splunk web interface may also be configured in your web.conf file. If you choose, to use an ECDHE cipher you must provide the elliptic curve name to be used. The option for this as stated in the splunk specification files is: ecdhCurveName = <string> * ECDH curve to use for ECDH key negotiation * We only support named curves specified by their SHORT name. * (see struct ASN1_OBJECT in asn1.h) * The list of valid named curves by their short/long names * can be obtained by executing this command: * $SPLUNK_HOME/bin/splunk cmd openssl ecparam -list_curves * Default is empty string. One commonly used setting is the following. ecdhCurveName = prime256v1 NOTE: Since Splunk web, and the Splunk daemon communicate with each other, you need to make sure that whichever cipherSuite and ecdhCurveName are set in web.conf will also be supported by the cipherSuite parameters used in server.conf. ... View more

With Enterprise Security version 3.3.0, you can now adjust the configuration script, rather than disable it. go to the Settings > Data Inputs page. Click on Configuration Checker. Modify the "Suppression string" parameter of confcheck_script_errors to exclude the items needing to be suppressed. Only do this if you are sure the script is exiting normally, and returning the referenced code value. ... View more