topic Splunk learned app should stop learning / too_small / dispatch.evaluate in Getting Data In

Splunk learned app should stop learning / too_small / dispatch.evaluate

FRoth — Thu, 03 Mar 2016 13:25:36 GMT

I defined a new input folder that receives gzipped server logs from a scp copy job on our servers.

inputs.conf

[monitor://F:\ssh_incoming\Logs]
disabled = false
recursive = false
host_regex = pl\-([^_]+)_
index = plserver
sourcetype = syslog

The problem is that the props.conf of the learned app grows and grows until dispatching of new searches takes longer and longer until no searches start at all. (Inspect Job shows that dispatch.evaluate takes more than ten minutes to complete)

/learned/local/props.conf

[pl-www1_20160303053001_sudo-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303053001_system-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303060001_crond-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303060001_sshd-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

So as long as logs keep incoming in that folder, the props.conf grows and grows. I even disabled the "learned" App but that didn't solve the problem. The file keeps growing.

I have also tried to set LEARN_SOURCETYPE to false in the props.conf definition for the "syslog" sourcetype.

[syslog]
CHARSET = latin-1
LEARN_SOURCETYPE = false

I am using Splunk 6.2.2 with Enterprise License.

Re: Splunk learned app should stop learning / too_small / dispatch.evaluate

ncsantucci — Mon, 28 Mar 2016 10:02:15 GMT

If you want to disable to learned app see this post:

https://answers.splunk.com/answers/77271/make-splunk-stop-learning-sourcetypes.html

Here is the relevant part of the post:

If you really would like to disable learning, edit $SPLUNK_HOME/etc/apps/learned/local/app.conf and make sure it says this:

[install]
state = disabled

Re: Splunk learned app should stop learning / too_small / dispatch.evaluate

risgupta_splunk — Tue, 29 Sep 2020 10:46:35 GMT

If all the sourcetypes are marked as {{-too_small}}.

For small files Splunk is unable to determine the type. This can be controlled by:

{noformat:title=props.conf}
[too_small]
PREFIX_SOURCETYPE = false
{noformat}

Could you please try to add a new entry in props.conf with below configuration and restart splunk:

[too_small]
PREFIX_SOURCETYPE = false

This above configuration will not grow the sourcetypes in learned app and .gz file will also read and forwarded by the splunk.