https://community.splunk.com/t5/Getting-Data-In/How-to-use-HTTP-Event-Collector-HEC-to-setup-an-app-to-log-to/m-p/301558#M56981 <P>You run HEC on a heavy forwarder. If you need to scale you run multiples behind a load balancer. How much the HFs hold in pipelines depends on how the admin configured the heavy forwarders. </P> <P>However most people just scale enough HEC capacity to handle the load. Doing things acknowledgements are hard to implement for the application developer making the logs.</P> <P><A href="http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/Data/UsetheHTTPEventCollector">http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/Data/UsetheHTTPEventCollector</A><BR /> <A href="http://dev.splunk.com/view/event-collector/SP-CAAAE7G">http://dev.splunk.com/view/event-collector/SP-CAAAE7G</A></P> <P><A href="https://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Protectagainstlossofin-flightdata">https://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Protectagainstlossofin-flightdata</A></P> Fri, 23 Feb 2018 19:43:13 GMT starcher 2018-02-23T19:43:13Z https://community.splunk.com/t5/Getting-Data-In/How-to-use-HTTP-Event-Collector-HEC-to-setup-an-app-to-log-to/m-p/301557#M56980 <P>I am trying to set up an app to log to Splunk but I have a few (basic) questions.<BR /> First I was just going to write the events to a log on disk and have Splunk continuously monitor the file but I would like to steer away from writing to disk if possible because we store personal information. I like the HTTP Event Collector option but I am concerned with what happens if there are network connectivity issues or some other downtime. I see that there is some resilience built into the appenders but does that mean that the events will be stored in some queue until a connection can be re-established? How big is that queue if there is one, and do events get dropped if the queue fills up? If I cant drop a single event log is this option even viable, or would it still be recommended to write to disk?</P> <P>Also talking with one of our sys admins he mentioned that he would have to set up a dedicated HEC forwarder, would this be the case? I thought that using the HEC option negated having to use a forwarder, or am I completely misunderstanding what needs to happen here to implement this?</P> Wed, 21 Feb 2018 20:03:34 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-use-HTTP-Event-Collector-HEC-to-setup-an-app-to-log-to/m-p/301557#M56980 mhelmers 2018-02-21T20:03:34Z https://community.splunk.com/t5/Getting-Data-In/How-to-use-HTTP-Event-Collector-HEC-to-setup-an-app-to-log-to/m-p/301558#M56981 <P>You run HEC on a heavy forwarder. If you need to scale you run multiples behind a load balancer. How much the HFs hold in pipelines depends on how the admin configured the heavy forwarders. </P> <P>However most people just scale enough HEC capacity to handle the load. Doing things acknowledgements are hard to implement for the application developer making the logs.</P> <P><A href="http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/Data/UsetheHTTPEventCollector">http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/Data/UsetheHTTPEventCollector</A><BR /> <A href="http://dev.splunk.com/view/event-collector/SP-CAAAE7G">http://dev.splunk.com/view/event-collector/SP-CAAAE7G</A></P> <P><A href="https://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Protectagainstlossofin-flightdata">https://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Protectagainstlossofin-flightdata</A></P> Fri, 23 Feb 2018 19:43:13 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-use-HTTP-Event-Collector-HEC-to-setup-an-app-to-log-to/m-p/301558#M56981 starcher 2018-02-23T19:43:13Z