Getting Data In

If an identical input is specified in inputs.conf for multiple apps on a universal forwarder, will this result in duplicate data?

JdeFalconr
Explorer

If an input is specified identically in the inputs.conf file of multiple apps running on a Universal forwarder, will the same data be gathered multiple times (and thus generate extra license usage) or just once?

I'm looking at indexed data volume in our Splunk environment for a particular sourcetype within the Splunk App for Microsoft Exchange (winhostmon). In looking at .conf files on one of our Exchange servers' Universal Forwarder, I see that the inputs.conf file within the "TA-Exchange-2010-CAS" app, the "TA-Exchange-2010-HUB" app, and the TA-Windows-2008R2-Exchange-IIS" app all have inputs for "winhostmon" defined precisely the same. None are disabled. Does that mean this data is being gathered and indexed once per application specifying this input? Or is the forwarder smart enough to only gather it once?

0 Karma

somesoni2
Revered Legend

The Splunk merges configurations/settings from all configurations files based on location-priority. If there are duplicates OR conflicts, it takes the settings from the copy with highest priority. So yes, the forwarder is smart enough that it will gather the data only once if same data input is specified at multiple app/places.

See this link for more information of how splunk configuration file precedence works.
http://docs.splunk.com/Documentation/Splunk/6.3.3/Admin/Wheretofindtheconfigurationfiles

Jeremiah
Motivator

Splunk consolidates overlapping input stanzas into a single input, so you should not see the same data indexed multiple times. Keep in mind that is for stanzas that overlap exactly. If there's a variation (ie, similar, but different file paths) then the behavior is different.

Take a look at this doc for information about order of precedence in conf files:

http://docs.splunk.com/Documentation/Splunk/6.1/admin/Wheretofindtheconfigurationfiles

You can list all of the active inputs on your system by using the btool command:

https://answers.splunk.com/answers/11259/how-do-i-list-all-active-input-stanzas-in-all-apps.html

And also with the command:

splunk list monitor

To verify the running configuration on the forwarder.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Developer Spotlight with Denis Gladkikh

From Splunk Engineer to Kubernetes App Builder Denis GladkikhWhat happens when a lifelong developer turns a ...

Governing Enterprise AI, Bringing Cisco Telemetry Home, and More from Splunk Lantern

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

Agentic SOC Triage: Investigating Splunk ES Notables with MCP Server and a Local LLM

The Problem: Too Many Alerts, Too Little Context Security operations teams running Splunk Enterprise Security ...