topic Re: Heavy forwarder with httpout to indexer cluster in Splunk Search

Heavy forwarder with httpout to indexer cluster

j01am — Mon, 19 Feb 2024 12:50:24 GMT

Hello everyone,

Quick question : I need to forward data from HF to Indexer cluster.

Right now, I'm using S2S tcpout function, with useAck, default loadbalancing and maxQueueSize

I study the possibility to use the httpout instead of tcpout, due to traffic filtering.

The documentation seems a bit light about httpout, is it possible to use Indexer loadbalancer, ack, and maxQueueSize function?

Thanks for your help!

Jonas

Re: Heavy forwarder with httpout to indexer cluster

j01am — Fri, 12 Apr 2024 22:33:48 GMT

Anyone have inputs about that?

Re: Heavy forwarder with httpout to indexer cluster

isoutamo — Sat, 13 Apr 2024 06:55:26 GMT

Hi
What issue you are trying to solve with this change?
I think that usually it’s better to use S2S between splunk nodes than http version.
r. Ismo

Re: Heavy forwarder with httpout to indexer cluster

j01am — Sat, 13 Apr 2024 10:14:00 GMT

Hello @isoutamo

Thanks a lot for your feedback

I need to study the httpout because Splunk nodes communicate though customer network, with firewalls, so it's easier to open a proxy compatible traffic than a tcp/9997 for example.

So, is there any possibility to use Indexer loadbalancer, ack, and maxQueueSize functions in httpout ?

A saw that httpout is a relative new functionnality, since 8.x, maybe these functionality will be in the roadmap?

Thanks

Jonas

Re: Heavy forwarder with httpout to indexer cluster

PickleRick — Sat, 13 Apr 2024 11:40:42 GMT

Disclaimer - I haven't used httpout much so I might be mistaken here.

But.

httpout is not a HEC output (although it needs an HEC input and valid HEC token; it's complicated). It's s2s protocol embedded in http transport. It is indeed a fairly recent invention mostly aimed at situations like yours - where it's easier (politically, not technically) to allow outgoing http traffic (even if it's only pseudo-http) than some unknown protocol.

Having said that I'd expect most of the functionalities normally working with tcpout (like useACK) to work.

I'd test it first in the lab before pushing to prod anyway.

Re: Heavy forwarder with httpout to indexer cluster

j01am — Sun, 14 Apr 2024 20:03:03 GMT

Current documentation is very light regarding httpout, hope it will be improved in next versions

Re: Heavy forwarder with httpout to indexer cluster

PickleRick — Sun, 14 Apr 2024 20:58:52 GMT

That's true. Remember that docs pages have feedback form on the bottom. You can use it to... provide feedback. And yes, this feedback (of course if precise and reasonable, not just "I don't like this page" ;-)) is read and the docs pages do get better in time because of that.