I am trying to set up an app to log to Splunk but I have a few (basic) questions.
First I was just going to write the events to a log on disk and have Splunk continuously monitor the file but I would like to steer away from writing to disk if possible because we store personal information. I like the HTTP Event Collector option but I am concerned with what happens if there are network connectivity issues or some other downtime. I see that there is some resilience built into the appenders but does that mean that the events will be stored in some queue until a connection can be re-established? How big is that queue if there is one, and do events get dropped if the queue fills up? If I cant drop a single event log is this option even viable, or would it still be recommended to write to disk?
Also talking with one of our sys admins he mentioned that he would have to set up a dedicated HEC forwarder, would this be the case? I thought that using the HEC option negated having to use a forwarder, or am I completely misunderstanding what needs to happen here to implement this?