topic Re: An issue with the HTTP Event Collector (HEC) has been identified in Splunk 7.x in Getting Data In

An issue with the HTTP Event Collector (HEC) has been identified in Splunk 7.x

jmaher_splunk — Thu, 26 Jul 2018 15:55:31 GMT

Summary:
After upgrading from Splunk Enterprise or Splunk Cloud 6.x to 7.x, customers are reporting a bug with HTTP Event Collector (HEC). As a result:

Some HEC events may not be getting ingested after the upgrade
There may be a reduction in performance (indexing throughput) related to HEC events.

What happened:
Splunk Enterprise and Splunk Cloud releases 7.x (“7.x”) include a limit on HTTP Event Collector (HEC) payloads of 512KB. This limit exists to prevent memory overuse. Post-7.0.x, HEC events with sizes exceeding 512KB are not resolved by the HEC parser, and may be dropped.

Which customers are impacted:
This issue may impact any customer meeting the following criteria:

Are on Splunk Enterprise or Splunk Cloud 7.x
Use HEC
Have a payload size above 512KB

Re: An issue with the HTTP Event Collector (HEC) has been identified in Splunk 7.x

jmaher_splunk — Thu, 26 Jul 2018 21:36:46 GMT

Resolution:

Splunk is working on a resolution to ensure the HEC module in Splunk Enterprise and Splunk Cloud 7.x is more tolerant of larger payloads by default, and we also plan to make the limit configurable to suit specific needs.
Splunk Cloud customers that are potentially impacted, will be contacted over the next few weeks to schedule a maintenance window
For Splunk Enterprise customers that are potentially impacted, this will be fixed in 7.0.5 (ETA July 27) and 7.1.3 (End of August). We will post to this thread as the maintenance releases are available.

Re: An issue with the HTTP Event Collector (HEC) has been identified in Splunk 7.x

jmaher_splunk — Sat, 28 Jul 2018 07:07:43 GMT

Update:

The latest maintenance release, 7.0.5, for Splunk Enterprise and Splunk UniversalForwarder are now available from the Download site.
Please note as 7.0.5 is not the latest version, you can find it under the “Older Releases” section.

Download: https://www.splunk.com/en_us/download.html
Known Issues: http://docs.splunk.com/Documentation/Splunk/7.0.5/ReleaseNotes/Knownissues
Fixed Issues: http://docs.splunk.com/Documentation/Splunk/7.0.5/ReleaseNotes/Fixedissues

Re: An issue with the HTTP Event Collector (HEC) has been identified in Splunk 7.x

shankern — Tue, 31 Jul 2018 05:53:39 GMT

Is gzip content encoding header supported on HEC ? Would be useful while posting large payloads.

Re: An issue with the HTTP Event Collector (HEC) has been identified in Splunk 7.x

adammike — Wed, 19 Jun 2019 21:57:35 GMT

and we also plan to make the limit configurable to suit specific needs

How do I configure this? I can't find anything in the docs or online

Re: An issue with the HTTP Event Collector (HEC) has been identified in Splunk 7.x

sylim_splunk — Thu, 20 Aug 2020 01:33:52 GMT

maxEventSize = <positive integer>[KB|MB|GB]
* The maximum size of a single HEC (HTTP Event Collector) event.

https://docs.splunk.com/Documentation/Splunk/8.0.5/Admin/Inputsconf