Getting Data In

Do Props.conf create any effect, in customize app at Forwrader?

Path Finder

Hey Splunkers!

I have a doubt, when we create any customize app in Splunk, for any purpose, lets say for log monitoring.
So the default props.conf will be effective or if i update something in my Customize App's props.conf at UF level, so that will be effective for my particular sourcetype?

As i read somewhere, If the sourcetype specified in the inputs.conf of Splunk UF was not declared in the props.conf in the Splunk Indexer of Splunk HF, the attributes of the sourcetype will take all the default props.conf settings (Line_BREAKER, TIME_FORMAT etc...) of Universal Forwarder.

Thanks in advance!
Keep Splunking

0 Karma
1 Solution

Builder

The Universal forwarder does minimal parsing of data:
https://docs.splunk.com/Documentation/Splunk/7.2.5/Forwarding/Typesofforwarders

To achieve higher performance and a lighter footprint, it has several limitations:

• The universal forwarder cannot search, index, or produce alerts with data.
• The universal forwarder does not parse data. You cannot use it to route data to different Splunk indexers based on its contents.
• Unlike full Splunk Enterprise, the universal forwarder does not include a bundled version of Python.

https://docs.splunk.com/Documentation/Splunk/7.2.5/Forwarding/Typesofforwarders#Types_of_forwarder_d...

With unparsed data, a universal forwarder performs minimal processing. It does not examine the data stream, but it does tag the stream with metadata to identify source, source type, and host. It also divides the data stream into 64-kilobyte blocks and performs some rudimentary timestamping on the stream that the receiving indexer can use in case the events themselves have no discernible timestamps. The universal forwarder does not identify, examine, or tag individual events except when you configure it to parse files with structure data (such as comma-separated value files.)

Recommend putting the props.conf on both UF and indexing tier.

View solution in original post

0 Karma

Splunk Employee
Splunk Employee

I highly recommend the Aplura cheat sheet Where do my Splunk props.conf settings belong?

0 Karma

Builder

The Universal forwarder does minimal parsing of data:
https://docs.splunk.com/Documentation/Splunk/7.2.5/Forwarding/Typesofforwarders

To achieve higher performance and a lighter footprint, it has several limitations:

• The universal forwarder cannot search, index, or produce alerts with data.
• The universal forwarder does not parse data. You cannot use it to route data to different Splunk indexers based on its contents.
• Unlike full Splunk Enterprise, the universal forwarder does not include a bundled version of Python.

https://docs.splunk.com/Documentation/Splunk/7.2.5/Forwarding/Typesofforwarders#Types_of_forwarder_d...

With unparsed data, a universal forwarder performs minimal processing. It does not examine the data stream, but it does tag the stream with metadata to identify source, source type, and host. It also divides the data stream into 64-kilobyte blocks and performs some rudimentary timestamping on the stream that the receiving indexer can use in case the events themselves have no discernible timestamps. The universal forwarder does not identify, examine, or tag individual events except when you configure it to parse files with structure data (such as comma-separated value files.)

Recommend putting the props.conf on both UF and indexing tier.

View solution in original post

0 Karma

Path Finder

Hey @solarboyz1 ,
Thanks much for your response,
My intent for asking this was,

Try to analyse 1 scenario, i have forwarder installed on a server, now in system\local\inputs.conf of that forwarder i have defined my monitoring path, index and sourcetype. Also, at system\local\props.conf, i have defined some attributes say Line Breaker, Time Stamp and rest other things, (i know its not advisable to make modification in system\local, but we need to monitor for only this particular server.)
Now when these logs are parsing to Indexer directly, lets consider, they are adapting the props.conf settings what we had mentioned at forwarder,(assuming, as i am restricted to use Indexer's UI). Now when the logs are moving from Indexer to Search Head, will they still be adapting the same setting, what it was on forwarder, or it will take the setting of default? As we have not mentioned our source type attributes on Indexer's props.conf.

0 Karma

Builder

So, the UF is sending data that hasn't really be parsed as much as metadata about source, soucetype, etc.. is sent with it. For example, the UF does not perform line breaking for events. It sends file content in chunks to the indexer.

The indexer is fully parsing the data, which includes line-breaking and index time extractions, this basically defines how individual events are identified and indexed. It takes the chunks of data, and the information about that data (the metadata sent with it from the forwarder) and runs the data through the parsing queue.

The search head does report time parsing, which is really just cosmetic, in that it can change how the event is displayed but does not change how the event was indexed.

So, to answer your question: Now when the logs are moving from Indexer to Search Head, will they still be adapting the same setting, what it was on forwarder, or it will take the setting of default?

No, the indexer will not use the settings from the forwarder. If a props.conf for this host, source, or sourcetype is not configured, then Splunk will use the default settings and attempt to learn this sourcetype.

The metadata sent by the forwarder is available (source, sourcetype, index) and will be used unless over-rode by props.conf defined on the indexer.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!