Hello,
We would like to use the following search as an input for the ML Toolkit Numeric Field Prediction (DecisionTreeRegressor):
index=mlbso violation sourcetype=*BWP_hanatraces* | timechart span=10s count as mlbso_hana_composite_ooms
The idea is to count the occurrences of the word "violation" (out of memory errors) in the specific source type, create the field "mlbso_hana_composite_ooms" out of it and then predict on it. Now, in order to improve the prediction quality, we need to run the above search for the time span long enough in the past, ideally some months. This however produces the following error:
The specified span would result in too many (>50000) rows
I understand where the error comes from (maxresultrows in limits.conf) but we do not want to extend the span. We would actually want event to reduce it to 1 sec. Neither do we want to make the time window smaller — also here we would rather extend it to several months.
When I reduce the selected time window or put the time span higher in order to stay below 50000 rows, then it works fine but brings around 3.000 events back. So, shortly speaking, out of 50.000 samples, I get 3.000 events where the word "violation" was found — with this, the quality of the DecisionTreeRegressor is not good enough.
How would I overcome this issue?
Am I overlooking something?
Kind Regards,
Kamil
