We have 3 clustered indexers and an original Search Head. Installed an app that has a custom props.conf on the Search Head, and it is NOT showing the extracted proper fields when performing searches. Deployed a new Search Head and installed the same exact app. The new Search Head shows the proper fields. The two servers appear to be identical, and running: splunk cmd btool props list --debug ...shows the same exact results, line by line, for the app. The original server does have some extra apps, but with the results of the btool above, it would not appear there are any conflicts with other apps. What would be the next steps in troubleshooting the original Search Head, and why it does not show the proper fields? ... View more

Here is the inputs.conf entry:   [batch://opt/splunk/var/run/splunk/csv/*.csv] disabled = false move_policy = sinkhole index = test-metrics sourcetype = metrics_csv     However, as I monitor /opt/splunk/var/run/splunk/csv/ I see the CSV files are still there, and not getting indexed. This should have been a really simple test, but can't figure out why batch is not working. If I hardcode a specific CSV file it works:   [batch://opt/splunk/var/run/splunk/csv/test.csv] disabled = false move_policy = sinkhole index = test-metrics sourcetype = metrics_csv     But obviously I need it to get all the CSV files, so I should be able to use the wildcard *.csv ... View more

Here is the sample set of data, simplified: Aug 8 11:00:00 host=host1 status_code=UP Aug 8 12:20:00 host=host1 status_code=UP Aug 8 14:15:00 host=host1 status_code=UP Aug 8 15:00:02 host=host1 status_code=DOWN Aug 8 15:05:02 host=host1 status_code=DOWN Aug 8 15:10:02 host=host1 status_code=DOWN Aug 8 15:15:02 host=host1 status_code=UP Aug 8 16:20:00 host=host1 status_code=UP Aug 8 16:50:00 host=host1 status_code=UP Basically, it is checking the host to be up (or down). If it notices the host is down, it will recheck within 5 minutes (and continues until it is UP again). It logs the UP at various times throughout the day. Original thought to get uptime would be this: status_code=DOWN OR status_code=UP host_name="host1" | sort host_name | eval HOSTUP=if(status_code="UP",1,0) | eval HOSTDOWN=if(status_code="DOWN",1,0) | eval UPTIME=(HOSTUP/(HOSTUP+HOSTDOWN))*100 | eval DOWNTIME=(HOSTDOWN/(HOSTUP+HOSTDOWN))*100 | stats avg(UPTIME) AS UPTIME avg(DOWNTIME) as DOWNTIME by host_name | eval UPTIME=round(UPTIME,2) | eval DOWNTIME =round(DOWNTIME ,2) However, that is not true uptime over the timed search. That is just averaging the number of UPs vs the number of DOWNs, and is an incorrect representation of true uptime. With the UPs being inconsistent, that will not work. So, I need to figure out how to calculate how long (based on the search time interval) the status_code equaled DOWN. From that, I could calculate the percentage (over time) the host was down, and thereby calculate the uptime percentage. I was attempting some transaction searches, but I can not seem to get the syntax correct. The end results should be host was down for 10 minutes out of the time search, let's say it was "Last 24hrs", which is only 0.1% (0.0069444e) of the total minutes (1440) of the day. Any suggestions? ... View more