We have a problem with parsing logic which generates hundreds of thousands of truncate warnings every hour:
06-21-2014 18:14:34.121 -0700 WARN LineBreakingProcessor - Truncating line because limit of 800 has been exceeded with a line length >= 6985 - data_source="/
this issue causes Indexers to choke and eventually create indexing lag.
can someone clarify how to disable indexing of LineBreakingProcessor Warnings in _internal index?
i have tried updating /app/share/splunk/etc/log.cfg category.TailingProcessor=INFO to ERROR but it didnt work.
You could explicitly tell Splunk to truncate the line, and then Splunk wouldn't warn you that it did...
Put the following in props.conf
# set the following to the maximum line length that you want
TRUNCATE = 800
Also, the TailingProcessor does not do line-breaking, so you might as well set that back to INFO. If you want to change the setting for the LineBreakingProcessor, edit the file named log-debug.cfg. You will need to uncomment the line.