topic Re: Sending input data over HTTP in Getting Data In

Sending input data over HTTP

justjosh — Thu, 27 Sep 2012 15:58:17 GMT

Does Splunk support receiving a continual stream of input via an HTTP POST?

The reason I ask is the web server logs I want to index in Splunk is not accessible from the Splunk Server. I already have a tool which can relay server log events over HTTP so I think this could work.

Re: Sending input data over HTTP

MarioM — Thu, 27 Sep 2012 16:12:15 GMT

you can pass the data directly via the Splunk Rest API:

Splunk Rest API

Re: Sending input data over HTTP

bmacias84 — Thu, 27 Sep 2012 17:18:25 GMT

@justjosh, Yes, you can send raw event on the wire directly to a TCP. They should be in best practice format, but it should handle them just fine. In your props you will have to define how to turn the stream into an event. Like MarioM mention earlier there are to REST endpoint call recievers/simple and receivers/stream.

http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTinput#receivers.2Fstream
http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTinput#receivers.2Fsimple
http://dev.splunk.com/view/logging-best-practices/SP-CAAADP6

Hope this helps.

Re: Sending input data over HTTP

rarsan_splunk — Thu, 22 Oct 2015 18:41:25 GMT

Yes. Refer to HTTP Event Collector newly introduced with Splunk 6.3
http://dev.splunk.com/view/event-collector/SP-CAAAE6M

Re: Sending input data over HTTP

Graham_Hanningt — Tue, 29 Sep 2020 09:37:39 GMT

Re:

Does Splunk support receiving a continual stream of input via an HTTP POST?

No.

Not a continual (endless) stream.

You can "batch" (send multiple) events in a single HTTP POST, but there is a maximum limit to the size of an HTTP request.

The limit is set by max_content_length in limits.conf. The default value, as of Splunk 6.4, is 1000000 bytes (~ 1 MB).

Exceeding that limit results in the HTTP response error code 413 (request entity too large).

The Splunk documentation that describes batching events (such as "About the JSON event protocol in HTTP Event Collector") does not mention this limit (at least, I can't find any such mention). I think it should.