Sending input data over HTTP

[justjosh]

Does Splunk support receiving a continual stream of input via an HTTP POST?

The reason I ask is the web server logs I want to index in Splunk is not accessible from the Splunk Server. I already have a tool which can relay server log events over HTTP so I think this could work.

[Graham_Hanningt]

Re:

Does Splunk support receiving a continual stream of input via an HTTP POST?

No.

Not a continual (endless) stream.

You can "batch" (send multiple) events in a single HTTP POST, but there is a maximum limit to the size of an HTTP request.

The limit is set by max_content_length in limits.conf. The default value, as of Splunk 6.4, is 1000000 bytes (~ 1 MB).

Exceeding that limit results in the HTTP response error code 413 (request entity too large).

The Splunk documentation that describes batching events (such as "About the JSON event protocol in HTTP Event Collector") does not mention this limit (at least, I can't find any such mention). I think it should.

[rarsan_splunk]

Yes. Refer to HTTP Event Collector newly introduced with Splunk 6.3
http://dev.splunk.com/view/event-collector/SP-CAAAE6M

[bmacias84]

@justjosh, Yes, you can send raw event on the wire directly to a TCP. They should be in best practice format, but it should handle them just fine. In your props you will have to define how to turn the stream into an event. Like MarioM mention earlier there are to REST endpoint call recievers/simple and receivers/stream.

http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTinput#receivers.2Fstream
http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTinput#receivers.2Fsimple
http://dev.splunk.com/view/logging-best-practices/SP-CAAADP6

Hope this helps.

[MarioM]

you can pass the data directly via the Splunk Rest API:

Splunk Rest API