Getting Data In

Why are we getting "Changing breaking behavior for event stream because MAX_EVENTS (256) was exceeded without a single event break"

mlevsh
Builder

Hi,

We have started to experience line breaking issue for our csv source. As a result sometimes we have an attempt from Splunk to read a whole cvs file with 500+ lines as one event up to 256 lines in it. Then these errors occur and Splunk starts reading the rest of the file correctly: one line per one event.

AggregatorMiningProcessor - Changing breaking behavior for event stream because MAX_EVENTS (256) was exceeded without a single event break. Will set BREAK_ONLY_BEFORE_DATE to False, and unset any MUST_NOT_BREAK_BEFORE or MUST_NOT_BREAK_AFTER rules. Typically this will amount to treating this data as single-line only. - data_source="/tmp/tmp-in/tmp/d_tmp_storage_history.csv", data_host="host06", data_sourcetype="d_tmp_storage_history"
host = hf_host  source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd

WARN  AggregatorMiningProcessor - Breaking event because limit of 256 has been exceeded - data_source="/tmp/tmp-in/tmp/d_tmp_storage_history.csv", data_host="host06", data_sourcetype="d_tmp_storage_history"

The issue seems to be started few days ago. No known changes were introduced on Splunk side or on the side that runs scripts to generate cvs files we monitor.

Here is props.conf we use on Splunk Universal Forwarder on data_host="host06" that monitors data_source="/tmp/tmp-in/tmp/d_tmp_storage_history.csv"

[ d_tmp_storage_history ]
HEADER_FIELD_LINE_NUMBER = 1
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
INDEXED_EXTRACTIONS=csv
KV_MODE=none
DATETIME_CONFIG=CURRENT

We have checked csv file via Excel ,Notepad+ ,vi editor for hidden characters (:set list), cat with -v -t -e - to see if some special unusual character(s) pop up. Haven't found anything unusual

Any advice which direction to look would be appreciated!
Thank you

0 Karma
1 Solution

mlevsh
Builder

I believe we found the issue: extra spaces in the stanaza's name before d_ and after y:
[ d_tmp_storage_history ]. After we deleted spaces and pushed the props.conf, it seemed to correct the issue

View solution in original post

0 Karma

mlevsh
Builder

I believe we found the issue: extra spaces in the stanaza's name before d_ and after y:
[ d_tmp_storage_history ]. After we deleted spaces and pushed the props.conf, it seemed to correct the issue

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...