Getting Data In

Is there a way to change collection interval for HTTP Event Collector?

splunkguy0342
New Member

I am using HTTP Event Collector to collect Symantec ATP logs, my current ingest rate varies based on log size. It is typically around 2000-5000 logs at a rate of every 1 minute. My log source is generating between 1.5 M -3 M events per day. The collector is averaging about 480k-960k events per day. This is putting me into a logging deficit where I am unable to keep up with log generation. I am looking to change the interval to every 5 seconds or vastly increase the collection rate. I am for the most part default settings, the event collector is running on a heavy forwarder and forwarding to an indexer cluster, we have tried pointing to a single indexer but performance did not change.

0 Karma

lguinn2
Legend

Where is the actual bottleneck on the heavy forwarder: network, memory, CPU?

Forwarding to an indexer cluster should not be slower than forwarding to a single indexer, so I am not surprised that didn't help.
There is no "collection interval" on the heavy forwarder; it should be able to "collect" the events asynchronously as they are sent over http/https.

My guess is that you may be exceeding the bandwidth of a single event collector. Have you considered using 2 heavy forwarders and having the sender switch between them?

If the resources on the heavy forwarder are not being taxed, then perhaps the sender trying to exceed its output bandwidth.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...