I have a cluster of Windows indexers. I need to backup my new warm buckets every day and cannot afford to wait until buckets roll to frozen to back them up. I was thinking of creating a script that would copy out my warm buckets right from my Splunk storage and send them to my cold storage system. However, I was warned that this could interfere with Splunk trying to read from those buckets while I am copying them. My second thought was to use VSS shadow copies. I can have a script that creates a VSS shadow copy of my storage drive on my indexers, and then copies the warm buckets from the shadow copy, instead of copying from the original data. But my understanding of VSS is that it also will prevent Splunk from reading any data while the shadow copy is being made. I have seen documentation referencing the use of VSS to back up warm buckets from Splunk, but nothing that goes into detail on how it works or how it should be implemented. Here are my questions: Is Splunk VSS aware? Will VSS interfere with Splunk reading data from the warm buckets? Am I right in thinking that copying buckets right from Splunk storage, with robocopy for example, will interfere with Splunk being able to read data? Is there any other recommended way to back up Splunk storage without waiting for the buckets to roll to frozen? ... View more

I have a bunch of buckets that I want to restore. According to documentation, the dirt step it finding the buckets you want to restore and then copying them to the $SPLUNK_HOME/var/lib/splunk/{INDEX}/thaweddb directory. Then you have to run the rebuild command. It is not clear in the documentation, however, whether or not you have to thaw the buckets on the same indexer that they came from. I am looking at http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/Restorearchiveddata I am running version 6.5.2 with an indexer cluster. First, the documentation says that on versions 4.2 and higher, you can thaw data on any indexer instance, not just the one that it originated on. "For the most part, you can restore an archive to any instance of the indexer, not just the one that originally indexed it. This, however, depends on a couple of factors:Splunk Enterprise version. You cannot restore a bucket created by Splunk Enterprise 4.2 or later to a pre-4.2 indexer. The bucket data format changed between 4.1 and 4.2, and pre-4.2 indexers do not understand the new format. This means: 4.2+ buckets: You can restore a 4.2+ bucket to any 4.2+ instance." Then, at the bottom of the page, it talks about restoring data in a clustered environment and it says that you should place the buckets in the thawed directory of the indexer that it originated on: "However, as described in "Archive indexed data", it is difficult to archive just a single copy of clustered data in the first place. If, instead, you archive data across all peer nodes in a cluster, you can later thaw the data, placing the data into the thawed directories of the peer nodes from which it was originally archived." Do I have to thaw buckets only on the indexer that the data origniated on? ... View more

I want to collect Event Hub data from Azure and I think using the Azure Monitor Add-on would be the best way to do it. There is some pretty good documentation, including a Wiki, on how to install and ocnfigure this add-on. However, nowhere does it say where this add-on should be installed. Does it get installed on the searchhead, heavy forwarder, indexers, or all of the above? Python and Node.js also need to be installed. Do they get installed on the same instances? ... View more

Hello, does anyone have any more information on SPL-154382? I am planning an upgrade to 7.0.5 and this is one of the known issues. The description reads, "Role Capability To See Indexes for Summary Indexing Gives Role Index Edit Ability." This sounds like if a user can view a summary index, they can also edit it. That seeme like a pretty big concern to me! ... View more

Hello, We want to send Azure Information Protection (AIP) data to Splunk. I have read the Splunk Docs on how to set up the Splunk Add-On for Microsoft Cloud Services to collect data, but it doesn't specify which services within Azure that it can collect data from. Does anyone know if the add-on can be used to collect from AIP? ... View more

Hey Everyone, I installed Splunk_TA_nix on my Ubuntu 16.04.2 server. After enabling some scripts and not seeing any data beng monitored, I checked splunkd.log and I see the following error: 07-03-2018 16:13:04.110 +0100 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin/cpu.sh" /bin/sh: 0: Can't open For some reason the UF cannot of the .sh script files. As shown below, Splunk is the owner of those files and it has execute permissions: -rwxrwxr-x 1 splunk splunk 3447 Jul 3 15:21 bandwidth.sh* -rwxrwxr-x 1 splunk splunk 3997 Jul 3 15:21 common.sh* -rwxrwxr-x 1 splunk splunk 3997 Jul 3 15:21 common.sh* Does anyone know what is wrong here? ... View more

I have installed the UF on a number of servers and I configured ti to monitor the winodws event logs (Application, System, Security). It looks like the UF has only picked up the event logs starting from when it was installed. Is there a way to tell the UF to ingest all of the event logs from the past? ... View more

Hello, Recently, I have been rexieving this error in python.log on my search head. As a result of the error, an email alert still gets sent out, but there is no PDF attached to it. Instead, it says "An error occurred while generating the PDF. Please see python.log for details." 2018-04-25 00:02:17,805 -0400 ERROR sendemail:1074 - An error occurred while generating a PDF: Failed to fetch PDF (status = 400): Unable to render PDF.&lt;br/&gt;<ul><li>Exception raised while trying to prepare "*" for rendering to PDF. Splunkd daemon is not responding: ("Error connecting to /servicesNS/**/search/parser: ('The read operation timed out',)",)</li></ul> ... View more

I have the following query that is inteded to divide the "stats.hypervisor_cpu_ppm" field by 10000 and then show that value in a table. index=nutanix sourcetype=nutanix_arch stats.hypervisor_cpu_usage_ppm=* | eval usage=stats.hypervisor_cpu_usage_ppm / 10000 | table host, stats.hypervisor_cpu_usage_ppm, usage | dedup host When I run the query, It gives me a table with the host, the values for stats.hypervisor_spu_ppm, and then an empty column for usage. Why is the usage column empty? ... View more

Hi, Is there a way to authenticate to the API through LDAP or SAML? right now, the only way I can authenticate is by using a local static account that I have configured to have API access. However, our security policy prohibits the use of local unmanaged accounts. I have SAML authentication configured for web access, but when I try to use those same AD credentials to authenticate to the API it does not work. Thanks! ... View more