I’m receiving an error whenever I try to view any csv lookup tables I have uploaded into my search head cluster (v8.1.6).   Uploading the same csv files on to my local sandbox works without issue.     With the query   | inputlookup <filename>.csv   I receive the error   The lookup table '<filename>.csv' requires a .csv or KV store lookup definition.   The .csv files appear on the local file system and propagate across the cluster properly.  The splunkd.log also doesn't give any information beyond what the UI already outputs. ... View more

We've added a self signed cert to our haproxy server which passes traffic on to our search head cluster. After doing so, I changed the following on web.conf [settings] tools.proxy.on = true tools.proxy.base = https://internalhost.local Now when trying to visit https://internalhost.local it properly maintains https and redirects to https://internalhost.local/en-US, which then redirects to https://127.0.0.1:8000. I can't seem to find any configuration value in my web or server settings that calls out 127.0.01 which puts me at a loss for what to adjust. ... View more

We're retiring our internally hosted Splunk environment and moving the data into an EC2 instance on AWS. It seems like our best solution is to use SmartStore and I'm trying to determine the best way to migrate our data. We're moving multiple TB of logs Once that data is in S3, we won't be adding any new logs to Splunk. We would like the old data searchable. We will be reducing our Indexer count for 7 down to 2 as this environment will be minimally accessed. I believe the best solution is to enable SmartStore on our servers and once the data is transferred to S3, create the new indexers and decommission our old environment. Am I missing something with this plan? ... View more

I'm having an issue running the command: splunk apply shcluster-bundle -target http://<cluster_captain>:8089 This yields the error: Error while deploying apps to first member: ConfDeploymentException: Error while fetching apps baseline on target=http://<cluster_captain>:8089: Network-layer error: Connection reset by peer After numerous attempts to change my pass4symkey under the shclustering stanza in my server.conf file, I'm still getting the same error. When we had originally set up the search head cluster, this worked properly, but I can't seem to find a way to debug this issue. Adding the -debug flag gives the following: (1ry) QUERYING: 'base_url:https://127.0.0.1:8089 auth_option:(null) relative_url:/services/apps/deploy command=POST' In build_full_rest_url(): Composing URL from base=https://127.0.0.1:8089 + relative=/services/apps/deploy In build_full_rest_url(): Composed URL=https://127.0.0.1:8089/services/apps/deploy In make_simple_rest_call_online(): using_basic_auth=0 In make_simple_rest_call_online(): [Re-]Initialized HTTP request headers: <none> In make_simple_rest_call_online(): HTTP request response_code=500 Online REST call to /services/apps/deploy returned -1 Error while deploying apps to first member: ConfDeploymentException: Error while fetching apps baseline on target=http://<cluster_captain>:8089: Network-layer error: Connection reset by peer (1ry) FAILED: 'HTTP/1.1 500 Error while deploying apps to first member: ConfDeploymentException: Error while fetching apps baseline on target=http://<cluster_captain>:8089: Network-layer error: Connection reset by peer' As I can restart all of my search heads in the cluster and it's able to reform a cluster, I don't believe this is a pass4symkey issue. Any suggestions what would cause this? ... View more

I have files on multiple servers that I need to log that are housed in a directory where the path includes the system's hostname. The issue I'm dealing with is I only want the first portion of the hostname. For example, if my hostname is set to db01.awesomesite.com , the path I'm attempting to monitor would be /var/log/stuff/db01/otherstuff.log I've attempted this to no avail: [monitor:///var/log/stuff/*/otherstuff.log] host_segment = 4 sourcetype = awesome_sourcetype index = awesome_index disabled = false Changing the host in the Universal Forwarder config isn't acceptable as I need to maintain the full host in my Splunk host field. ... View more