Getting Data In

What’s the recommended batch size per request for sending to HEC?

spammenot66
Contributor

Is there a limit to how many events can be sent to Splunk HEC per event? What’s recommended, are there any guideline 


This Splunk conf has it at 5-50, but I’ve seen some folks send 1k-6k events per request? Is there a point where # of events per request starts to affect performance and would it affect just the input with large request or the overall HEC server?

https://conf.splunk.com/files/2017/slides/measuring-hec-performance-for-fun-and-profit.pdf

“Recommendation: Batch size between 5 and 50“

Labels (1)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

I have done performance testing in the last 12 months using a HEC buffer size of 256K and then various sizes of up to 5MB. Using an event size of approx 170 bytes. That equates to approx 740 messages per buffer (allowing for additional HEC metadata sent with the request), with the 5MB buffer size up to 15K events per batch.

A lot will depend on how many events you are generating, but just in terms of getting data into Splunk, the number of events per batch doesn't seem to affect performance. If you have very large data volumes, then there are other performance settings you will need to manage, as referred to in that CONF link. I have tested with 3 parallel pipelines and 8 dedicated IO threads, across a 6 index cluster with rates of 900 events/sec (~256MB/sec)

That conf PDF says 5-50 for batch size, but later says 100 events/request. so not sure what it's actually saying.

Note that timing will play a part also. If you have 1 event per second, then if you only send the HEC payload after 100 events, then you will get index lag of 100 seconds in your event stream, so factor that in.

 

View solution in original post

0 Karma

bowesmana
SplunkTrust
SplunkTrust

I have done performance testing in the last 12 months using a HEC buffer size of 256K and then various sizes of up to 5MB. Using an event size of approx 170 bytes. That equates to approx 740 messages per buffer (allowing for additional HEC metadata sent with the request), with the 5MB buffer size up to 15K events per batch.

A lot will depend on how many events you are generating, but just in terms of getting data into Splunk, the number of events per batch doesn't seem to affect performance. If you have very large data volumes, then there are other performance settings you will need to manage, as referred to in that CONF link. I have tested with 3 parallel pipelines and 8 dedicated IO threads, across a 6 index cluster with rates of 900 events/sec (~256MB/sec)

That conf PDF says 5-50 for batch size, but later says 100 events/request. so not sure what it's actually saying.

Note that timing will play a part also. If you have 1 event per second, then if you only send the HEC payload after 100 events, then you will get index lag of 100 seconds in your event stream, so factor that in.

 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...