Getting below error when click in Test button on Jenkins server

[asifkh11]

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /jenkins/descriptorByName/com.splunk.splunkjenkins.SplunkJenkinsInstallation/testHttpInput.
clicking on this link above shows this below error
URI does not specify a valid host name: http:/services/collector/event

Configs below
splunk host name = ip
Http input port = 8088
Http input token = 7176AE07-BC14-4171-8011-5F6D0FBA7875
SSL not enabled

[araza_splunk]

Hi,
Are you using Splunk Cloud ?

[txiao_splunk]

From the error message "The proxy server could not handle the request POST /jenkins/descriptorByName/com.splunk.splunkjenkins.SplunkJenkinsInstallation/testHttpInput ", I guess you are running Jenkins behind Proxy, could you check the reverse proxy follow the guide https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+says+my+reverse+proxy+setup+is+broken?
see also
https://wiki.jenkins-ci.org/display/JENKINS/Running+Jenkins+behind+Apache
https://wiki.jenkins-ci.org/display/JENKINS/Running+Jenkins+behind+Nginx

[asifkh11]

So our setup is splunk enterprise, with distributed cluster 2 sites and Shclustering as well, so the data input has been set on a seperate server we use fro all datainput,

[asifkh11]

Araza in metata data just one line when comented out works "#console_log.index=jenkins_console"

source=Jenkins
sourcetype=default-hudson.model.Executor
host=jenkins.availity.com

#custom index, uncomment the settings after index created in splunk
index=airtools
build_report.index=jenkins
file.index=jenkins_artifact
#console_log.index=jenkins_console

#send config content to splunk when user is making changes
jenkins_config.monitoring=true
jenkins_config.enabled=true

[asifkh11]

Since this is internal network so no proxy in between
I can test connection and it comes back fine "Splunk connection verified", it only breaks if i comment below line

console_log.index=jenkins_console,

with that line commented out connection works but no data is ingested in splunk

[txiao_splunk]

your console log will goto airtools (if the index exists) after "console_log.index=jenkins_console" commented.
You need create index in Splunk manually before update corresponding settings in Jenkins, the Splunk for Jenkins plugin will not create index for you.

Did you get warning message in Splunk such like
Received event for unconfigured/disabled/deleted index=airtools
Received event for unconfigured/disabled/deleted index=jenkins_console

[asifkh11]

Looks like it is not even hitting the splunk from Jenkins, so no msgs like above
I did create index=airtools, do i also need to create the 2 index below as well ?
build_report.index=jenkins
console_log.index=jenkins_console

[txiao_splunk]

I can see you set default source type to customzed "default-hudson.model.Executor" which may affect console logs, can you try add
console_log.sourcetype=httpevent

[txiao_splunk]

Or you can comment sourcetype=default-hudson.model.Executor to let Splunk select source type based on content.

[asifkh11]

No error, No msgs for recieved events, none of the Jenkin apps dashboard filing in

[txiao_splunk]

you need create index jenkins, jenkins_artifact, jenkins_statistics and jenkins_console, as the App expects a certain metadata format, here is the settings for the app

index=jenkins_statistics
build_report.index=jenkins
file.index=jenkins_artifact
console_log.index=jenkins_console

For more information, please check https://wiki.jenkins-ci.org/display/JENKINS/Splunk+Plugin+for+Jenkins