topic Re: Splunk heavy forwarding, how to index logs on receiving end in Getting Data In

Splunk heavy forwarding, how to index logs on receiving end

balajivs — Tue, 26 Oct 2021 09:53:08 GMT

Hi,
I have configured Splunk heavy forwarder in 2 machines. I want to send logs from one machine to another and expect the receiver to store all the received logs in an index called "receivedlogs".

This is the video I followed to configure Splunk: https://www.youtube.com/watch?v=S4ekkH5mv3E&t=454s&ab_channel=Splunk%26MachineLearning

Thank you.

Re: Splunk heavy forwarding, how to index logs on receiving end

gcusello — Tue, 26 Oct 2021 10:15:28 GMT

Hi @balajivs,

let me understand:

you have two machines, both configured as Heavy Forwarders (HF1 and HF2),
HF1 has to send logs to HF2, is it correct?
I can suppose that you have also at least another machine configured as Indexer that will finally receive logs, is it correct?
Does HF2 locally stores a copy of data or does it forwards all the data to the Indexer?

Anyway, if HF1 locally inputs logs, you can configure the index in the inputs.conf file.

If instead HF1 receive and forwards logs, you have to configure selective indexing and forwarding in HF1 as described at https://docs.splunk.com/Documentation/Splunk/8.2.2/Forwarding/Routeandfilterdatad#Perform_selective_indexing_and_forwarding

Ciao.

Giuseppe

Re: Splunk heavy forwarding, how to index logs on receiving end

balajivs — Tue, 26 Oct 2021 11:25:44 GMT

Hi @gcusello ,
On machine 1 have configured inputs.conf such a way that it will monitor locally stored logs. The output.conf is configured to send those locally stored logs to machine 2's port 9997. Machine 2 is listening to port 9997 and it by default stores the logs received on index "main".
I want to store logs received from 9997 port to a specific index called "receivedlogs". I tried going through the documentation you mentioned but I was unable to find a proper solution.

Thank you.

Re: Splunk heavy forwarding, how to index logs on receiving end

gcusello — Tue, 26 Oct 2021 11:37:29 GMT

Hi @balajivs,

I try to translate:

HF1 is the log source,
if you have only to take logs, you don't need an Heavy Forwarder (that as it's named it's an heavy monitoring systema), probably you could use an Universal Forwarder that's lighter;
HF2, isn't an Heavy Forwarder but an Indexer.

Now, you have two choices to define the index:

on the first machine, you could insert the index definition in the inputs;
on the second machine, you could use the selective indexing I described in my previous answer.

I hint to define index in the first machine, you can do this in many ways:

editing inputs.conf on the first machine and adding "index = receivedlogs" in the stanza related to the log input;
otherwise, you can open the web gui at the page you used to configure input and in the middle of the window there's a field "index" where you can define the index to store logs, if you don't see the index listed in the dropdown, you have to use the other solution or create a local empty index with the same name.

Ciao.

Giuseppe

Re: Splunk heavy forwarding, how to index logs on receiving end

PickleRick — Tue, 26 Oct 2021 11:48:47 GMT

Also, correct me if I'm wrong, if OP choses to write the messages localy to an index on the indexer and also forward the to another splunk instance where they will get separately indexed, the indexed events will consume the license twice - once on the intermediate indexer, once on the destination indexer.