topic Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers? in Getting Data In

How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

nirmalya2006 — Fri, 16 Dec 2016 12:21:05 GMT

Hi All

We currently have universal forwarder installed in our 3 application servers to forward application logs to Indexer.
The inputs.conf file in each of the application server looks like this

[monitor://C:\logs\logfiles\Application\Applog_*]
sourcetype = business_iis
index = business_idx1

The outputs.conf file in each of the application server looks like this

[tcpout:LoadBalancedIndexers]
defaultGroup = LoadBalancedIndexers
server = splunkbusinessindexer.info.com:13071

We are trying to implement the concept of Intermediate forwarder for the 3 application servers.
We will have an intermediate universal Splunk forwarder which will receive the log files from the universal Splunk forwarders installed in each application servers and forward them to Indexer.

For that I am trying to configure the inputs.conf and outputs.conf files in the Application servers and the Intermediate forwarder.
I am not able to understand which IP and port number should be configured in which file in comparison to what we already have.

Can someone please help me in writing the correct configuration.

Thanks
Nirmalya

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

skalliger — Fri, 16 Dec 2016 13:02:09 GMT

Just curious: What exactly do you mean by "intermediate"? Are you using an additional universal forwarder or are you using a heavy forwarder? Just asking because I don't see a reason for an additional universal forwarder acting as in between.

Regarding your question:
Basically, you have to configure these things:

1) Change the outputs.conf on your application server universal forwarder. These ones point to your intermediate forwarder with hostname and port, something like this

[tcpout]
indexAndForward = false
defaultGroup = YourIntermediateForwarder

[tcpout:YourIntermediateForwarder]
server = YourIntermediateForwarder:9997

And maybe additional settings if you're using SSL.

2) Specify an inputs.conf on your intermediate forwarder in system/local to listen on that specified port (e.g. 9997), something like this:

[default]
host = YourIntermediateForwarder

[splunktcp:9997]
compressed = true
disabled = 0
connection_host = none    <-- set this only if you have specified the host in your outputs.conf

3) Now specfy the outputs.conf on your intermediate forwarder to point to your indexers, like your actual outputs.conf:

[indexAndForward]
index = false

[tcpout:LoadBalancedIndexers]
defaultGroup = LoadBalancedIndexers
server = splunkbusinessindexer.info.com:13071

Did that help you?

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

nirmalya2006 — Fri, 16 Dec 2016 13:10:43 GMT

Thanks a lot. I was really looking for some idea on what entry to put in which file.
I will try these configurations and see if it works.

Regarding your query why I am using this.
There is some constraints on public IP addresses. So instead of using 3 public IP addresses ion the application server to connect to Indexer, we want to use 1 public IP address on the Intermediate Forwarder. The 3 Application servers can have private IP addresses to forward the logs to the Intermediate forwarder.
And yes I am using Universal forwarder for Intermediate forwarder.

Thanks again for the suggestions. Let me try configuring.

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

skalliger — Fri, 16 Dec 2016 15:32:54 GMT

And did you manage to make your configuration work?

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

nirmalya2006 — Sat, 17 Dec 2016 13:11:17 GMT

The admin team was not available on Friday. Will make the config on Monday. I am also eagerly waiting to make the config work. 😞

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

nirmalya2006 — Tue, 20 Dec 2016 13:42:48 GMT

@skalliger
Thank you very much. The configuration worked like a charm in the first attempt.

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

skalliger — Tue, 20 Dec 2016 13:51:20 GMT

Glad to hear that! 🙂 Thanks for accepting the answer.

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

ademianczuk — Tue, 22 Aug 2017 20:34:45 GMT

Great explanation thanks skalliger. To remain consistent with the documentation in v6.6 the splunktcp stanza should contain the foreslash escape character:
e.g.,

[splunktcp://9997]

This might have been a recent change since your reply was originally posted though 🙂

Re: How to configure an Intermediate Forwarder and the inputs.conf and outputs.conf files in the Application servers?

hectorvp — Mon, 13 Jul 2020 18:30:34 GMT

This answer helped me a lot as well, thanks.

I've a more requirement over this, that is I need to route data to different splunk enterprise instances based on host, how can I achieve this at Intermediate forwarder, when Intermediate forwarder is a Universal Forwarder?

listening on a single port, which is getting mixed data from two different hosts, I needs to send data from one host to its splunk enterprise and other hosts data to its splunk enterprise.

Few business and third party systems are the reasons who are creating such requirements, I knw it isn't a right practice, but then I need to, I would be really happy If i get answers at earliest