topic Using Heavy Forwarders as an intermediary Layer in Getting Data In

Using Heavy Forwarders as an intermediary Layer

dimitris_vergos — Tue, 23 Sep 2014 07:07:15 GMT

Hello,

I am currently doing a Splunk implementation where I have multiple Universal Forwarders which will be sending information to my Heavy Forwarders, where we will be doing a lot of filtering (thus is why we choose to have HF in between).

Thus the flow of events as of now is the following: UF -> HF -> Indexer

I have two questions as of now:

1.When it comes to TAs for Windows (even for Linux), do I have to place them both on the UF and HF (ofcourse I have to put them on the SH as well) or does it suffice if I put them just on the UF?

2.When information is coming in my UF into my HF I want them to go to a particular index. From what I have tested by just adding in the inputs.conf file the following:

HF - inputs.conf

[splunktcp://9997]
index = os_index

it wont' put the information into the index i want

Second attempt that i did was try to also modify the inputs.conf at the indexer for that particular HF:

[splunktcp://HF1:9997]
index = os_index

but still no luck.

What is the best method to do it?

I have to add routing information on the HF to push it to the correct index?

If so, do I need to also deploy the Windows/Linux TAs on the HF as well?

Re: Using Heavy Forwarders as an intermediary Layer

MarioM — Tue, 23 Sep 2014 08:00:07 GMT

Regarding 1.

inputs.conf should be in the UF anything else in the HF or Indexer and SH

Regarding 2.

This wont work as if you want data to go to a specific index you need to either:

set it at inputs.conf level in your uf with index= but not for splunktcp://
or use transforms at the heavyforwarder or indexer based on host,source,sourcetype or any specific string on the event itself: http://docs.splunk.com/Documentation/Splunk/6.1.3/Indexer/Setupmultipleindexes#Send_events_to_specific_indexes

Re: Using Heavy Forwarders as an intermediary Layer

dimitris_vergos — Mon, 28 Sep 2020 17:39:43 GMT

Yes, you are right, played around with it a bit and now it is working byt adding in the inputs.conf the necessary index.

Now my questions is the following Regarding Point one.

I have a deployment server, and I have deployed the Splunk_TA_Windows application to my UFs (which contain my local folder with the inputs.conf, outputs.conf file, and other files/folders such as props.conf etc.).

Now for my HF I should create a different application on my deployment server that will include all Splunk_TA_Windows files (excluding inputs.conf and outputs.conf, since they are being managed by a different application) is that correct?

Also for the Search Head, since it will not be doing any receiving of data, do I have to modify anything and create a local directory for the TA_Windows or leave it as is @ $SPLUNK_HOME/etc/apps with its default directory?

Re: Using Heavy Forwarders as an intermediary Layer

MarioM — Mon, 28 Sep 2020 17:39:45 GMT

the easiest is to just deploy Splunk_TA_Windows to all and have disabled inputs.conf in local where you dont need to collect the data

Re: Using Heavy Forwarders as an intermediary Layer

dimitris_vergos — Tue, 23 Sep 2014 10:43:34 GMT

Thanks MarioM