Getting Data In
Highlighted

Splunk learned app should stop learning / too_small / dispatch.evaluate

Contributor

I defined a new input folder that receives gzipped server logs from a scp copy job on our servers.

inputs.conf

[monitor://F:\ssh_incoming\Logs]
disabled = false
recursive = false
host_regex = pl\-([^_]+)_
index = plserver
sourcetype = syslog

The problem is that the props.conf of the learned app grows and grows until dispatching of new searches takes longer and longer until no searches start at all. (Inspect Job shows that dispatch.evaluate takes more than ten minutes to complete)

/learned/local/props.conf

[pl-www1_20160303053001_sudo-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303053001_system-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303060001_crond-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303060001_sshd-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

So as long as logs keep incoming in that folder, the props.conf grows and grows. I even disabled the "learned" App but that didn't solve the problem. The file keeps growing.

I have also tried to set LEARN_SOURCETYPE to false in the props.conf definition for the "syslog" sourcetype.

[syslog]
CHARSET = latin-1
LEARN_SOURCETYPE = false

I am using Splunk 6.2.2 with Enterprise License.

Highlighted

Re: Splunk learned app should stop learning / too_small / dispatch.evaluate

Path Finder

If you want to disable to learned app see this post:

https://answers.splunk.com/answers/77271/make-splunk-stop-learning-sourcetypes.html

Here is the relevant part of the post:

If you really would like to disable learning, edit $SPLUNK_HOME/etc/apps/learned/local/app.conf and make sure it says this:

[install]
state = disabled

0 Karma
Highlighted

Re: Splunk learned app should stop learning / too_small / dispatch.evaluate

Splunk Employee
Splunk Employee

If all the sourcetypes are marked as {{-too_small}}.

For small files Splunk is unable to determine the type. This can be controlled by:

{noformat:title=props.conf}
[toosmall]
PREFIX
SOURCETYPE = false
{noformat}

Could you please try to add a new entry in props.conf with below configuration and restart splunk:

[toosmall]
PREFIX
SOURCETYPE = false

This above configuration will not grow the sourcetypes in learned app and .gz file will also read and forwarded by the splunk.