Getting Data In

Configuration file precedence on universal forwarder and indexer

stwong
Communicator

Hi all,

We set sourcetype in inputs.conf on universal forwarder, e.g.

[monitor:///Firewall/*/*_pa_firewall.log]
ignoreOlderThan=1d
disabled = false
host_segment = 2
index = network
sourcetype = pan:log
no_appending_timestamp = true

Sourcetype of related logs changed to pan:traffic. Found that it's caused by an add-on defined on indexer that transforms the sourcetype for a matched pattern.

Then configuration file on indexer is of higher priority than those on universal forwarder. Is that correct?

Thanks a lot.
/st wong

0 Karma
1 Solution

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

View solution in original post

0 Karma

ragedsparrow
Contributor

Kind of. There are pipelines and queues that data travels through before it is stored. At a high level, Universal Forwarders are only able to the input layer in that it will read the data and add some metadata to it (like index, sourcetype, source) and then send it to the indexer for the data to go through the rest of the process.

So, it's not that it has precedence or is higher priority, it just does that function because the Universal Forwarder can't. If you had a scenario where you were using a Heavy Forwarder, you could perform that functionality on the Heavy Forwarder and the configurations on the indexer wouldn't apply to the data at that point.

Some resources:

0 Karma

stwong
Communicator

Thanks for your help.

0 Karma
Get Updates on the Splunk Community!

Demo Day: Strengthen Your SOC with Splunk Enterprise Security 8.1

Today’s threat landscape is more complex than ever. Security operation centers (SOCs) are overwhelmed with ...

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...