topic Re: HEC general purpose perf configs in Getting Data In

What are the best HEC perf tuning configs?

hrawat — Thu, 10 Aug 2023 21:15:51 GMT

What are the best HEC perf tuning configs?

Re: What are the best HEC perf tuning configs?

hrawat — Sun, 29 Jun 2025 19:04:53 GMT

inputs.conf 
[http] 
dedicatedIoThreads = 8
busyKeepAliveIdleTimeout = 300 
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerHandshakeTimeout = 300 
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection.
# Supported from M-release for cloud and on-prem 8.2.0)

On-prem Splunk version 9.4.0 and above can also use 
auto pipeline feature that scales up queues and pipelinesets instead of hardcoded values
Server.conf
[general]
autoAdjustQueue = true
pipelineSetAutoScale = true


server.conf (upto 9.3.x)
[general] 
parallelIngestionPipelines = 2 
#(never set more than 3 on indexing tier. However on SH/HF it can be set upto number of cores)

#Avoid 503 response back, have enough queue buffer for spike in ingestion

[queue=indexQueue]
maxSize = 100MB

[queue=aggQueue]
maxSize = 100MB

[queue=parsingQueue]
maxSize = 100MB #Must for HEC

[queue=httpInputQ]
maxSize = 100MB OR <10% of persistent queue size if persistent queue enabled> ##Must for HEC.

[queue=rulesetQueue]
maxSize = 100MB

[queue=typingQueue]
maxSize = 100MB

limits.conf 
[input_channels]
max_inactive = 10000 
#( ideally 2 times max(new_channels))

[input_channels] 
lowater_inactive = 9000 
#( max_inactive -1000 recommended if max_inactive > 10000)

indexes.conf 
[<PER_INDEX>] 
maxTimeUnreplicatedNoAcks=60 
#( needed if useACK=false and indexer cluster environment)

Note: Don't set maxSize(server.conf) or queueSize(inputs.conf) more than 10% of persistentQueueSize( if persistent queue is enabled)

Re: HEC general purpose perf configs

gjanders — Thu, 10 Aug 2023 07:32:37 GMT

Just on inputs.conf you have:

inputs.conf 
[http] 
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerSessionTimeout = 300

However I found that sslServerSessionTimeout appears to be in server.conf

Is that the wrong conf file?

Re: HEC general purpose perf configs

hrawat — Thu, 10 Aug 2023 15:48:19 GMT

Good catch. It was meant to be sslServerHandshakeTimeout. Will fix my answer.

Re: HEC general purpose perf configs

gjanders — Fri, 11 Aug 2023 05:39:56 GMT

Definitely an improvement, however the sslServerHandshakeTimeout is in server.conf so perhaps the answer should advise that?
It looks like it refers currently to inputs.conf

Re: HEC general purpose perf configs

hrawat — Fri, 11 Aug 2023 10:21:17 GMT

HEC specific config is in inputs.conf

https://docs.splunk.com/Documentation/Splunk/latest/admin/inputsconf

Re: HEC general purpose perf configs

gjanders — Fri, 11 Aug 2023 10:49:21 GMT

Ok I see it now. The previous setting was in server.confirm but this setting is in inputs.conf

thanks