How to set Splunk Cloud HTTP-INPUT Truncating to 10000 Characters

[CS_]

Hi all,

We are using Splunk Cloud, and I am using the https://http-inputs-mydomain.com/services/collector/raw to send a log file for ingestion.

The problem is that each line in this log file can be quite big, 25000 characters or more.

Splunk Cloud is truncating at 10,000 characters.

I can find steps for handling this on Splunk On-Prem for Heavy Log Forwarders etc. but doesn't seem to be addressed for the http-inputs on cloud.

Any idea's on how I can change it to accept larger logs?

Thanks,

Chris

[CS_]

We were able to get it working by amending the source type on the HEC token, and setting the character limit to 1mb (1,000,000 characters)

[tshah-splunk · Accepted solution]

Hey @CS_,

Can you try defining the TRUNCATE parameter in the props.conf file for the sourcetype that you have defined in the inputs.conf for HEC Token input. The default value of this parameter is 10000, hence the reason for truncating at 10000 characters. You can increase the value to accept larger logs.

More details for the parameter can be found here - https://docs.splunk.com/Documentation/Splunk/8.2.4/Admin/Propsconf 

[CS_]

thanks @tshah-splunk I'll have a look and see if I can make that change. will update with result.

-C

[VatsalJagani]

Yes, the TRUNCATE parameter should definitely work as you are using services/collector/raw endpoint for HEC.

Just to keep in mind, the TRUNCATE parameter or any other index-time/parsing parameters will not work if you are using just the regular /services/collector endpoint.