Getting Data In

What are the best HEC perf tuning configs?

hrawat_splunk
Splunk Employee
Splunk Employee

What are the best HEC perf tuning configs?

Labels (3)
Tags (2)
0 Karma
1 Solution

hrawat_splunk
Splunk Employee
Splunk Employee

 

inputs.conf 
[http]
dedicatedIoThreads = 8
busyKeepAliveIdleTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerHandshakeTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection.
# Supported from M-release for cloud and on-prem 8.2.0)

server.conf
[general]
parallelIngestionPipelines = 2
#(never set more than 3 on indexing tier. However on SH/HF it can be set upto number of cores)
#Avoid 503 response back, have enough queue buffer for spike in ingestion
[queue=indexQueue]
maxSize = 100MB
[queue=aggQueue]
maxSize = 100MB
[queue=parsingQueue]
maxSize = 100MB #Must for HEC
[queue=httpInputQ]
maxSize = 100MB #Must for HEC
[queue=rulesetQueue]
maxSize = 100MB
[queue=typingQueue]
maxSize = 100MB
limits.conf 
[input_channels]
max_inactive = 10000
#( ideally 2 times max(new_channels))

[input_channels]
lowater_inactive = 9000
#( max_inactive -1000 recommended if max_inactive > 10000)

indexes.conf
[<PER_INDEX>]
maxTimeUnreplicatedNoAcks=60
#( needed if useACK=false and indexer cluster environment)
 
 

View solution in original post

Tags (1)
0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

Good catch. It was meant to be sslServerHandshakeTimeout. Will fix my answer.

0 Karma

gjanders
SplunkTrust
SplunkTrust

Definitely an improvement, however the sslServerHandshakeTimeout is in server.conf so perhaps the answer should advise that?
It looks like it refers currently to inputs.conf

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee
0 Karma

gjanders
SplunkTrust
SplunkTrust

Ok I see it now. The previous setting was in server.confirm but this setting is in inputs.conf

 

thanks

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

 

inputs.conf 
[http]
dedicatedIoThreads = 8
busyKeepAliveIdleTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerHandshakeTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection.
# Supported from M-release for cloud and on-prem 8.2.0)

server.conf
[general]
parallelIngestionPipelines = 2
#(never set more than 3 on indexing tier. However on SH/HF it can be set upto number of cores)
#Avoid 503 response back, have enough queue buffer for spike in ingestion
[queue=indexQueue]
maxSize = 100MB
[queue=aggQueue]
maxSize = 100MB
[queue=parsingQueue]
maxSize = 100MB #Must for HEC
[queue=httpInputQ]
maxSize = 100MB #Must for HEC
[queue=rulesetQueue]
maxSize = 100MB
[queue=typingQueue]
maxSize = 100MB
limits.conf 
[input_channels]
max_inactive = 10000
#( ideally 2 times max(new_channels))

[input_channels]
lowater_inactive = 9000
#( max_inactive -1000 recommended if max_inactive > 10000)

indexes.conf
[<PER_INDEX>]
maxTimeUnreplicatedNoAcks=60
#( needed if useACK=false and indexer cluster environment)
 
 
Tags (1)
0 Karma

gjanders
SplunkTrust
SplunkTrust

Just on inputs.conf you have:

inputs.conf 
[http]
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerSessionTimeout = 300

However I found that sslServerSessionTimeout appears to be in server.conf

Is that the wrong conf file?

0 Karma
Get Updates on the Splunk Community!

Combine Multiline Logs into a Single Event with SOCK - a Guide for Advanced Users

This article is the continuation of the “Combine multiline logs into a single event with SOCK - a step-by-step ...

Everything Community at .conf24!

You may have seen mention of the .conf Community Zone 'round these parts and found yourself wondering what ...

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...