All Apps and Add-ons
Highlighted

Why is the data not getting displayed in Splunk from Jenkins?

Engager

I am getting the below error in Jenkins after adding Splunk plugin, and data is also not getting uploaded to Splunk from Jenkins.

May 10, 2018 11:06:20 AM com.splunk.splunkjenkins.utils.LogConsumer run
SEVERE: message not delivered:{length:1687 {"host":"","index":"jenkins_statistics","sourcetype":"json:jenkins","time":"1525946780.049","source":"jenkins","event":{"queue_time" ...}
com.splunk.splunkjenkins.utils.LogConsumer$SplunkClientError: Bad Request, incorrect index or invalid data format, status code:400
    at com.splunk.splunkjenkins.utils.LogConsumer$1.handleResponse(LogConsumer.java:68)
    at com.splunk.splunkjenkins.utils.LogConsumer$1.handleResponse(LogConsumer.java:40)
    at shaded.splk.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:222)
    at shaded.splk.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:164)
    at shaded.splk.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:139)
    at com.splunk.splunkjenkins.utils.LogConsumer.run(LogConsumer.java:90)
0 Karma
Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

SplunkTrust
SplunkTrust

Does the index jenkins_statistics exist on your system?

0 Karma
Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

Engager

Yes, I see the INDEX present on my Splunk server and its in enabled state also.

0 Karma
Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

SplunkTrust
SplunkTrust

Did you blank the "host":"" information or is it actually missing?

cheers, MuS

Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

Engager

Hi Mus,

I am not sure of this option, can you help me to find where it will be?

0 Karma
Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

Splunk Employee
Splunk Employee

the HEC token doesn't have write access to jenkins_statistics, please config the token
http://dev.splunk.com/view/event-collector/SP-CAAAE7F

0 Karma
Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

Explorer

Since I've upgraded Splunk plugin on one of the masters I am seeing the same errors(logs are flooded with them) but only on specific builds:

Mar 25, 2020 4:43:51 PM com.splunk.splunkjenkins.utils.LogConsumer run
SEVERE: message not delivered:{length:62626 {"host":"xxxxxx","index":"jenkins_console","sourcetype":"text:jenkins","time":"1585154629.437","source":"job/xxxxxxx/job/master/55 ...}
com.splunk.splunkjenkins.utils.LogConsumer$SplunkClientError: Bad Request, incorrect index or invalid data format, status code:400
        at com.splunk.splunkjenkins.utils.LogConsumer$1.handleResponse(LogConsumer.java:68)
        at com.splunk.splunkjenkins.utils.LogConsumer$1.handleResponse(LogConsumer.java:40)
        at shaded.splk.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:223)
        at shaded.splk.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
        at shaded.splk.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)
        at com.splunk.splunkjenkins.utils.LogConsumer.run(LogConsumer.java:89)

Any idea what could be causing this?

0 Karma
Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

Explorer

I've actually noticed this happening on other masters too and it is generating error messages every second or so which is impacting the Jenkins servers' performance.
@txiao_splunk has anyone else reported this issue with the latest version of the Splunk plugin?

0 Karma
Highlighted

Re: Why is the data not getting displayed in Splunk from Jenkins?

Splunk Employee
Splunk Employee

looks like a config issue.
the HEC token is either incorrect or does not have write access to jenkinsconsole, or jenkinsconsole index doesn't exist at all

0 Karma