Getting Data In

What are the best HEC perf tuning configs?

hrawat_splunk
Splunk Employee
Splunk Employee

What are the best HEC perf tuning configs?

Labels (3)
Tags (2)
0 Karma
1 Solution

hrawat_splunk
Splunk Employee
Splunk Employee

 

inputs.conf 
[http]
dedicatedIoThreads = 8
busyKeepAliveIdleTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerHandshakeTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection.
# Supported from M-release for cloud and on-prem 8.2.0)

server.conf
[general]
parallelIngestionPipelines = 2
#(never set more than 3 on indexing tier. However on SH/HF it can be set upto number of cores)
#Avoid 503 response back, have enough queue buffer for spike in ingestion
[queue=indexQueue]
maxSize = 100MB
[queue=aggQueue]
maxSize = 100MB
[queue=parsingQueue]
maxSize = 100MB #Must for HEC
[queue=httpInputQ]
maxSize = 100MB #Must for HEC
[queue=rulesetQueue]
maxSize = 100MB
[queue=typingQueue]
maxSize = 100MB
limits.conf 
[input_channels]
max_inactive = 10000
#( ideally 2 times max(new_channels))

[input_channels]
lowater_inactive = 9000
#( max_inactive -1000 recommended if max_inactive > 10000)

indexes.conf
[<PER_INDEX>]
maxTimeUnreplicatedNoAcks=60
#( needed if useACK=false and indexer cluster environment)
 
 

View solution in original post

Tags (1)
0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

Good catch. It was meant to be sslServerHandshakeTimeout. Will fix my answer.

0 Karma

gjanders
SplunkTrust
SplunkTrust

Definitely an improvement, however the sslServerHandshakeTimeout is in server.conf so perhaps the answer should advise that?
It looks like it refers currently to inputs.conf

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee
0 Karma

gjanders
SplunkTrust
SplunkTrust

Ok I see it now. The previous setting was in server.confirm but this setting is in inputs.conf

 

thanks

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

 

inputs.conf 
[http]
dedicatedIoThreads = 8
busyKeepAliveIdleTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerHandshakeTimeout = 300
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection.
# Supported from M-release for cloud and on-prem 8.2.0)

server.conf
[general]
parallelIngestionPipelines = 2
#(never set more than 3 on indexing tier. However on SH/HF it can be set upto number of cores)
#Avoid 503 response back, have enough queue buffer for spike in ingestion
[queue=indexQueue]
maxSize = 100MB
[queue=aggQueue]
maxSize = 100MB
[queue=parsingQueue]
maxSize = 100MB #Must for HEC
[queue=httpInputQ]
maxSize = 100MB #Must for HEC
[queue=rulesetQueue]
maxSize = 100MB
[queue=typingQueue]
maxSize = 100MB
limits.conf 
[input_channels]
max_inactive = 10000
#( ideally 2 times max(new_channels))

[input_channels]
lowater_inactive = 9000
#( max_inactive -1000 recommended if max_inactive > 10000)

indexes.conf
[<PER_INDEX>]
maxTimeUnreplicatedNoAcks=60
#( needed if useACK=false and indexer cluster environment)
 
 
Tags (1)
0 Karma

gjanders
SplunkTrust
SplunkTrust

Just on inputs.conf you have:

inputs.conf 
[http]
#(useful when HEC clients are using connection pools and want to keep connections idle. Set it 2 times expected idle time of connection. )
sslServerSessionTimeout = 300

However I found that sslServerSessionTimeout appears to be in server.conf

Is that the wrong conf file?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...