Getting Data In

Configuration file precedence on universal forwarder and indexer

stwong
Communicator

Hi all,

We set sourcetype in inputs.conf on universal forwarder, e.g.

[monitor:///Firewall/*/*_pa_firewall.log]
ignoreOlderThan=1d
disabled = false
host_segment = 2
index = network
sourcetype = pan:log
no_appending_timestamp = true

Sourcetype of related logs changed to pan:traffic. Found that it's caused by an add-on defined on indexer that transforms the sourcetype for a matched pattern.

Then configuration file on indexer is of higher priority than those on universal forwarder. Is that correct?

Thanks a lot.
/st wong

0 Karma
1 Solution

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

View solution in original post

0 Karma

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

0 Karma

stwong
Communicator

Thanks for your help.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Introducing the 2026 - 2027 SplunkTrust cohort!

The goal of the SplunkTrust™ membership has historically been to acknowledge and recognize those who go above ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...