Getting Data In

Indexer cluster ignores TRUNCATE for splunk_python

NoSpaces
Contributor

Have a nice day, everyone!
For continuous event truncation tracking, I have a simple alert that notifies me about truncation:

index="_internal" sourcetype="splunkd" log_level="WARN" "Truncating"
| rex field=_raw "limit\s+of\s+(?<Limit>\d+)\s"
| rex field=_raw "length\s+>=\s+(?<MessageLength>\d+)\s+"
| rex field=_raw "data_host=\"(?<Host>.*?)\",\s"
| rex field=_raw "data_sourcetype=\"(?<Sourcetype>.*?)\""
| stats count by Sourcetype, Host, Limit, MessageLength
| top limit=5 Sourcetype, MessageLength
| table Sourcetype, MessageLength

After my last update, I have started to notice that the 'splunk_python' sourcetype hits the default TRUNCATE value a couple of times per hour.
Overall, the events look fine, so I decided to tune the TRUNCATE value on the indexer cluster layer (manager-apps\_cluster\local\props.conf):

[splunk_python]
SHOULD_LINEMERGE = false
TRUNCATE = 20000

Unfortunately, it also didn't help, and I'm still getting alerts with the default message length (10000). Then I set TRUNCATE to 0 and found no difference. What am I doing wrong?

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Make sure you're setting the TRUNCATE value on the Host that is reporting the truncation.  Be sure that Splunk instance is restarted after making the change.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Make sure you're setting the TRUNCATE value on the Host that is reporting the truncation.  Be sure that Splunk instance is restarted after making the change.

---
If this reply helps you, Karma would be appreciated.
0 Karma

NoSpaces
Contributor

Your point is right.
I have also thought that truncation can appear only on IDXC layer but it were my SHCs.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Classic British Rock Bands

This puzzle series (first published here) is based on extracting information from statements about well-known ...

New to the Community? Join Us for an Exclusive Post .conf26 Welcome Session (Plus a ...

Keep the .conf26 Energy Going: Welcome to the Splunk Community! Whether we scanned your badge at the Community ...

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Register to Attend   Join our Tech Talk ...