Getting Data In

What are the best HEC perf tuning configs?

hrawat_splunk
Splunk Employee
Splunk Employee

What are the best HEC perf tuning configs?

Labels (3)
Tags (2)
0 Karma
1 Solution

hrawat_splunk
Splunk Employee
Splunk Employee

 

inputs.conf 
[http]
dedicatedIoThreads = 8
busyKeepAliveIdleTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerHandshakeTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection.
# Supported from M-release for cloud and on-prem 8.2.0)

server.conf
[general]
parallelIngestionPipelines = 2
#(never set more than 3 on indexing tier. However on SH/HF it can be set upto number of cores)
#Avoid 503 response back, have enough queue buffer for spike in ingestion
[queue=indexQueue]
maxSize = 100MB
[queue=aggQueue]
maxSize = 100MB
[queue=parsingQueue]
maxSize = 100MB #Must for HEC
[queue=httpInputQ]
maxSize = 100MB OR <10% of persistent queue size if persistent queue enabled> ##Must for HEC.
[queue=rulesetQueue]
maxSize = 100MB
[queue=typingQueue]
maxSize = 100MB
limits.conf 
[input_channels]
max_inactive = 10000
#( ideally 2 times max(new_channels))

[input_channels]
lowater_inactive = 9000
#( max_inactive -1000 recommended if max_inactive > 10000)

indexes.conf
[<PER_INDEX>]
maxTimeUnreplicatedNoAcks=60
#( needed if useACK=false and indexer cluster environment)
 
Note: Don't set maxSize(server.conf) or queueSize(inputs.conf) more than 10% of persistentQueueSize( if persistent queue is enabled)

View solution in original post

Tags (1)

hrawat_splunk
Splunk Employee
Splunk Employee

Good catch. It was meant to be sslServerHandshakeTimeout. Will fix my answer.

0 Karma

gjanders
SplunkTrust
SplunkTrust

Definitely an improvement, however the sslServerHandshakeTimeout is in server.conf so perhaps the answer should advise that?
It looks like it refers currently to inputs.conf

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee
0 Karma

gjanders
SplunkTrust
SplunkTrust

Ok I see it now. The previous setting was in server.confirm but this setting is in inputs.conf

 

thanks

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

 

inputs.conf 
[http]
dedicatedIoThreads = 8
busyKeepAliveIdleTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerHandshakeTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection.
# Supported from M-release for cloud and on-prem 8.2.0)

server.conf
[general]
parallelIngestionPipelines = 2
#(never set more than 3 on indexing tier. However on SH/HF it can be set upto number of cores)
#Avoid 503 response back, have enough queue buffer for spike in ingestion
[queue=indexQueue]
maxSize = 100MB
[queue=aggQueue]
maxSize = 100MB
[queue=parsingQueue]
maxSize = 100MB #Must for HEC
[queue=httpInputQ]
maxSize = 100MB OR <10% of persistent queue size if persistent queue enabled> ##Must for HEC.
[queue=rulesetQueue]
maxSize = 100MB
[queue=typingQueue]
maxSize = 100MB
limits.conf 
[input_channels]
max_inactive = 10000
#( ideally 2 times max(new_channels))

[input_channels]
lowater_inactive = 9000
#( max_inactive -1000 recommended if max_inactive > 10000)

indexes.conf
[<PER_INDEX>]
maxTimeUnreplicatedNoAcks=60
#( needed if useACK=false and indexer cluster environment)
 
Note: Don't set maxSize(server.conf) or queueSize(inputs.conf) more than 10% of persistentQueueSize( if persistent queue is enabled)
Tags (1)

gjanders
SplunkTrust
SplunkTrust

Just on inputs.conf you have:

inputs.conf 
[http]
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerSessionTimeout = 300

However I found that sslServerSessionTimeout appears to be in server.conf

Is that the wrong conf file?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...