HTTP 503 status code from HEC during high load

keio_splunk — Fri, 12 Jul 2019 01:48:15 GMT

Rsyslogd server is setup to send syslog messages to Splunk HTTP Event Collector (HEC) using omhttp module. During high load, HEC sending the following response to rsyslogd server: http status code: 503 reply: {"text":"Server is busy", "code":9, "invalid-event-number":0}. How to address this issue?

Re: HTTP 503 status code from HEC during high load

keio_splunk — Fri, 12 Jul 2019 01:52:00 GMT

HEC input queue is full and is telling Rsyslogd server that it is busy.

The HEC queueSize has to be increased to cope with the syslog messages:
1. Set the queueSize under the [http://] stanzas in $SPLUNK_HOME/etc/system/local/input.conf. Note: indicates the token name as assigned by the user, apply to individual tokens. Settings specified here override settings specified within the [http] stanza.
2. Restart the Splunk server.

Refer to http://dev.splunk.com/view/event-collector/SP-CAAAE6Q.

topic HTTP 503 status code from HEC during high load in Getting Data In

HTTP 503 status code from HEC during high load

Re: HTTP 503 status code from HEC during high load