Getting Data In

Configuration file precedence on universal forwarder and indexer

stwong
Communicator

Hi all,

We set sourcetype in inputs.conf on universal forwarder, e.g.

[monitor:///Firewall/*/*_pa_firewall.log]
ignoreOlderThan=1d
disabled = false
host_segment = 2
index = network
sourcetype = pan:log
no_appending_timestamp = true

Sourcetype of related logs changed to pan:traffic. Found that it's caused by an add-on defined on indexer that transforms the sourcetype for a matched pattern.

Then configuration file on indexer is of higher priority than those on universal forwarder. Is that correct?

Thanks a lot.
/st wong

0 Karma
1 Solution

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

View solution in original post

0 Karma

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

0 Karma

stwong
Communicator

Thanks for your help.

0 Karma
Get Updates on the Splunk Community!

Unlock New Opportunities with Splunk Education: Explore Our Latest Courses!

At Splunk Education, we’re dedicated to providing top-tier learning experiences that cater to every skill ...

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...