Getting Data In

Deploying an app to read CSV files, why is the Universal Forwarder is only processing settings from system/default/props.conf?

curtisb1024
Path Finder

I'm trying to deploy an app to a Universal Forwarder for reading CSV files, the problem is that none of the settings I'm trying to apply in the app via props.conf are being picked up. But if I add my sourcetype to system/default/props.conf (or modify a sourcetype already contained in the file), it appears to get picked up just fine.

I'm not using the built in csv sourcetype because I need to use some custom settings. Here's a stripped down example of my configs and the issue.

Props.conf settings

/etc/apps/myapp/default/props.conf
[TestSourceType1]
SHOULD_LINEMERGE = False
pulldown_type = true 
INDEXED_EXTRACTIONS = csv
KV_MODE = none
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"

/etc/system/default/props.conf
[TestSourceType2]
SHOULD_LINEMERGE = False
pulldown_type = true 
INDEXED_EXTRACTIONS = csv
KV_MODE = none
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"

BTool output

C:\Program Files\SplunkUniversalForwarder\bin>splunk cmd btool props list TestSourceType
[TestSourceType1]
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
HEADER_MODE =
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 10000
detect_trailing_nulls = auto
maxDist = 100
priority =
sourcetype =
[TestSourceType2]
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
HEADER_MODE =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = False
TRANSFORMS =
TRUNCATE = 10000
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
detect_trailing_nulls = auto
maxDist = 100
priority =
pulldown_type = true
sourcetype =

C:\Program Files\SplunkUniversalForwarder\bin>splunk cmd btool --app=myapp props list TestSourceType
[TestSourceType1]

Notice that TestSourceType1 has only the default settings applied to it, it's not picked up any of the settings from props.conf in the myapp app.

This source seems to suggest that what I'm trying to do should work, but no matter what I do I simply cannot get the UF to pick up settings from anything other than system/default/props.conf

Is there some setting I've missed? What am I doing wrong here?

I'm running Splunk 6.2 on my Indexer, and I've tried UF versions 6.1.1, 6.2, and 6.2.2.

0 Karma
1 Solution

curtisb1024
Path Finder

The root cause of this issue ended up being some non-standard line breaks in the props.conf file. Splunk, for whatever reason, was able to read the sourcetype name regardless of the line breaks and where the stanza was located within the props.conf, but was ignoring all setting under the sourcetype.

View solution in original post

0 Karma

curtisb1024
Path Finder

The root cause of this issue ended up being some non-standard line breaks in the props.conf file. Splunk, for whatever reason, was able to read the sourcetype name regardless of the line breaks and where the stanza was located within the props.conf, but was ignoring all setting under the sourcetype.

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...