Getting Data In

Why are we getting "Changing breaking behavior for event stream because MAX_EVENTS (256) was exceeded without a single event break"

mlevsh
Builder

Hi,

We have started to experience line breaking issue for our csv source. As a result sometimes we have an attempt from Splunk to read a whole cvs file with 500+ lines as one event up to 256 lines in it. Then these errors occur and Splunk starts reading the rest of the file correctly: one line per one event.

AggregatorMiningProcessor - Changing breaking behavior for event stream because MAX_EVENTS (256) was exceeded without a single event break. Will set BREAK_ONLY_BEFORE_DATE to False, and unset any MUST_NOT_BREAK_BEFORE or MUST_NOT_BREAK_AFTER rules. Typically this will amount to treating this data as single-line only. - data_source="/tmp/tmp-in/tmp/d_tmp_storage_history.csv", data_host="host06", data_sourcetype="d_tmp_storage_history"
host = hf_host  source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd

WARN  AggregatorMiningProcessor - Breaking event because limit of 256 has been exceeded - data_source="/tmp/tmp-in/tmp/d_tmp_storage_history.csv", data_host="host06", data_sourcetype="d_tmp_storage_history"

The issue seems to be started few days ago. No known changes were introduced on Splunk side or on the side that runs scripts to generate cvs files we monitor.

Here is props.conf we use on Splunk Universal Forwarder on data_host="host06" that monitors data_source="/tmp/tmp-in/tmp/d_tmp_storage_history.csv"

[ d_tmp_storage_history ]
HEADER_FIELD_LINE_NUMBER = 1
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
INDEXED_EXTRACTIONS=csv
KV_MODE=none
DATETIME_CONFIG=CURRENT

We have checked csv file via Excel ,Notepad+ ,vi editor for hidden characters (:set list), cat with -v -t -e - to see if some special unusual character(s) pop up. Haven't found anything unusual

Any advice which direction to look would be appreciated!
Thank you

0 Karma
1 Solution

mlevsh
Builder

I believe we found the issue: extra spaces in the stanaza's name before d_ and after y:
[ d_tmp_storage_history ]. After we deleted spaces and pushed the props.conf, it seemed to correct the issue

View solution in original post

0 Karma

mlevsh
Builder

I believe we found the issue: extra spaces in the stanaza's name before d_ and after y:
[ d_tmp_storage_history ]. After we deleted spaces and pushed the props.conf, it seemed to correct the issue

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...