Getting Data In

What are the precedence of stanza and option in props.conf

robertosegantin
Path Finder

Hi all,

I'm monitoring a set of logs using Splunk input.
By default they have the sourcetype "others-sourcetype" and using a transform I generate, for some of them, the source type "my-sourcetype" which is generated dinamically.

Splunk reads "others-sourceytpe" props.conf stanza and it applies only "TRANSFORMS-sourcetype_override", which change correctly the sourcetype "others-sourceytpe" into "my-sourcetype", but it ignore all other configurations: LINE_BREAKER , SEDCMD-blfRemover and so on.
Also, even if Splunk change "others-sourceytpe" into "my-sourcetype" it does not use again props.conf to read "my-sourcetype" stanza, so Splunk doesn't apply any event manipulation options

Can you help me to understand which is the precedence of the props.conf stanzas and options?

props.conf:

[others-sourcetype]
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
CHARSET = UTF-8
disabled = false
TRUNCATE = 100000
LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d+\s+\w+)
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%4N
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 24
SEDCMD-blfRemover = s/\x0A//g
SEDCMD-acrRemover = s/\x0D//g
TRANSFORMS-giorno=add_giorno
TRANSFORMS-sourcetype_override = others-sourcetype_logs_override
category = Custom
pulldown_type = 1


[my-sourcetype]
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
CHARSET = UTF-8
disabled = false
TRUNCATE = 100000
LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d+\s+\w+)
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%4N
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 24
SEDCMD-blfRemover = s/\x0A//g
SEDCMD-acrRemover = s/\x0D//g
TRANSFORMS-anno = anno

transforms.conf:

[others-sourcetype_logs_override]
REGEX = \/home\/user\/Desktop\/(.*)\/.*.log
SOURCE_KEY = MetaData:Source
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

[add_giorno]
REGEX = (?<giorno>^\d\d\d\d-\d\d-\d\d)
FORMAT = giorno::$1
WRITE_META = true

[anno]
REGEX = (?<anno>^\d\d\d\d)
FORMAT = anno::$1
WRITE_META = true

[mese]
REGEX =  (?<mese>^\d\d\d\d-\d\d)
FORMAT = mese::$1
WRITE_META = true
0 Karma
1 Solution

FrankVl
Ultra Champion

Splunk will indeed not process props again after changing the sourcetype.

What you can do is put the sourcetype overwriting transforms into a source based stanza. As far as I understand from the props.conf spec that should take priority.

View solution in original post

0 Karma

FrankVl
Ultra Champion

Splunk will indeed not process props again after changing the sourcetype.

What you can do is put the sourcetype overwriting transforms into a source based stanza. As far as I understand from the props.conf spec that should take priority.

0 Karma

robertosegantin
Path Finder

I tried also with source based stanza, but it does not work.
So Splunk read props.conf only once.

0 Karma

FrankVl
Ultra Champion

Did you try with source based for everything, or just for the sourcetype rewriting?

PS: taking another look at your transforms: since the sourcetype is purely based on the folder, the simple solution for this is of course to use separate input stanzas for each folder, assigning the correct sourcetype immediately.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...