Getting Data In

Splunk learned app should stop learning / too_small / dispatch.evaluate

FRoth
Contributor

I defined a new input folder that receives gzipped server logs from a scp copy job on our servers.

inputs.conf

[monitor://F:\ssh_incoming\Logs]
disabled = false
recursive = false
host_regex = pl\-([^_]+)_
index = plserver
sourcetype = syslog

The problem is that the props.conf of the learned app grows and grows until dispatching of new searches takes longer and longer until no searches start at all. (Inspect Job shows that dispatch.evaluate takes more than ten minutes to complete)

/learned/local/props.conf

[pl-www1_20160303053001_sudo-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303053001_system-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303060001_crond-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[pl-www1_20160303060001_sshd-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

So as long as logs keep incoming in that folder, the props.conf grows and grows. I even disabled the "learned" App but that didn't solve the problem. The file keeps growing.

I have also tried to set LEARN_SOURCETYPE to false in the props.conf definition for the "syslog" sourcetype.

[syslog]
CHARSET = latin-1
LEARN_SOURCETYPE = false

I am using Splunk 6.2.2 with Enterprise License.

risgupta_splunk
Splunk Employee
Splunk Employee

If all the sourcetypes are marked as {{-too_small}}.

For small files Splunk is unable to determine the type. This can be controlled by:

{noformat:title=props.conf}
[too_small]
PREFIX_SOURCETYPE = false
{noformat}

Could you please try to add a new entry in props.conf with below configuration and restart splunk:

[too_small]
PREFIX_SOURCETYPE = false

This above configuration will not grow the sourcetypes in learned app and .gz file will also read and forwarded by the splunk.

ncsantucci
Path Finder

If you want to disable to learned app see this post:

https://answers.splunk.com/answers/77271/make-splunk-stop-learning-sourcetypes.html

Here is the relevant part of the post:

If you really would like to disable learning, edit $SPLUNK_HOME/etc/apps/learned/local/app.conf and make sure it says this:

[install]
state = disabled

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...