Getting Data In

Configuration file precedence on universal forwarder and indexer

stwong
Communicator

Hi all,

We set sourcetype in inputs.conf on universal forwarder, e.g.

[monitor:///Firewall/*/*_pa_firewall.log]
ignoreOlderThan=1d
disabled = false
host_segment = 2
index = network
sourcetype = pan:log
no_appending_timestamp = true

Sourcetype of related logs changed to pan:traffic. Found that it's caused by an add-on defined on indexer that transforms the sourcetype for a matched pattern.

Then configuration file on indexer is of higher priority than those on universal forwarder. Is that correct?

Thanks a lot.
/st wong

0 Karma
1 Solution

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

View solution in original post

0 Karma

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

0 Karma

stwong
Communicator

Thanks for your help.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...