Getting Data In

How to increase the event size which is limited to 10000 in Splunk cloud ?

vasutiwari
New Member

Hi,
I am not able to fetch the full JSON payload using the scripted input in the Splunk cloud.
Here, I have installed a universal forwarder that is connected to Splunk cloud, then I have created a simple app that will run the python script. The script will give the API output in the JSON format. While fetching the event in the search app I am not able to see my full JSON payload which is truncating at the 10000 characters where my payload is having more than 30000 characters.

As per the suggestions from the Splunk answer which we change the limits.conf and props.conf files in the /system/default
still not able to sort out.

Please come up with solutions.
Thank you.

0 Karma

willemjongeneel
Communicator

Hello,

If you use Splunk Cloud Managed, you can make this change in Splunk Cloud through the web interface.

Settings > Sourcetypes > Select the Sourcetype you are using > click the Advanced tab > here the TRUNCATE 10000 option appears. If you change this to a higher value, this will probably solve your issue.

However as @nickhillscpl suggested, if you use a heavy forwarder then you'll probably have to make the change on the heavy forwarder.

Kind regards,
Willem

nickhills
Ultra Champion

To change truncation limits you need to apply this at the first parsing stage.

If you have UF>HF>Splunk Cloud then you need to make the change on the HF.
If you have UF>Splunk Cloud, then you need the change made on the indexers.

As a general note, you should NEVER make a change in system/default!
Changes should only be in system/local - but preferably in $SPLUNK_HOME/etc/apps/YOURAPPNAME/[default|local]

You say you have created an app - presumably which you push onto your UFs?
You should add the stanza into props.conf in that app (in /default if you maintain the app, in /local if its a third party app) - It wont take any effect on your UFs, but you can deploy the same app (with the inputs disabled) on your HF/indexers.

in props.conf

[yoursourcetype]
TRUNCATE = 0
If my comment helps, please give it a thumbs up!
0 Karma

jkat54
SplunkTrust
SplunkTrust

Did you submit a support ticket to have the limits increased? If not, that's what you want to do because the settings need to be made on the indexers and maybe the search heads too. I forget, but either way you need a support ticket I believe.

0 Karma

manjunathmeti
Champion

Set TRUNCATE = 0 in props.conf for your sourcetype.

[_json]
TRUNCATE = 0 
0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...