Log ingestion Via HEC - Huge log volume

VijaySrrie — Thu, 03 Dec 2020 04:54:14 GMT

Hi All,

We are ingesting huge volume of logs from fluentd to splunk via HEC method. Will there be any loss as huge volumes are ingested (5GB) per day?

If yes, how to rectify it?

Is there any alternate method to ingest fluentd logs?

Re: Log ingestion Via HEC - Huge log volume

livehybrid — Tue, 09 Feb 2021 04:14:59 GMT

What size host is your HEC receiver running? Its worth checking out https://www.splunk.com/pdfs/technical-briefs/splunk-validated-architectures.pdf to check your setup aligns with best practice, however I would not personally consider 5GB to be to a particularly large volume when it comes to Splunk HEC ingest.