topic Re: Forwardeing and Indexing on an Heavy Forwarder in Getting Data In

Forwardeing and Indexing on an Heavy Forwarder

gcusello — Mon, 30 Nov 2015 16:49:45 GMT

Hi at all,
I have a Splunk instance indexing some logs.
I'd like to continue to use the server for its old job but, at the same time, to use the same server (the same Splunk instance or a different one) to forward another log flow to a different Indexer without local indexing.
In other words: I have to locally index some flows and forward to another Indexer a different one.
I know that I can configure outputs.conf to forward logs to different indexers, but is it possible to send a flow to another Indexer and locally index other flows?
Can I do this with a single Splunk instance or do I have to install another Splunk instance (Universal or Heavy) to forward?
Thank you in advance.
Bye.
Giuseppe

Re: Forwardeing and Indexing on an Heavy Forwarder

ltrand — Mon, 30 Nov 2015 17:20:50 GMT

You'll need a new instance as you can only have one tcpout stanza per instance and that stanza is where you set indexAndForward. So yes, but you have to use two Splunk instances. If you can provide more detail on the situation then better guidance can be given, but I would suggest to use the UF to forward off the log that you don't want to index locally but want to forward remotely.

Re: Forwardeing and Indexing on an Heavy Forwarder

jeffland — Tue, 01 Dec 2015 07:04:43 GMT

Your single splunk instance can do both at the same time, see here for docs.

Basically, you can set up any forwarding and/or indexing settings for each input stanza independently.

Re: Forwardeing and Indexing on an Heavy Forwarder

jeffland — Tue, 01 Dec 2015 07:07:25 GMT

No. You can have more than one tcpout stanza, and you can assign these to different inputs. Also, indexAndForward is not the only setting that governs indexing and forwarding; you can route any input any way you like, its called selective indexing. See these docs for more.

Re: Forwardeing and Indexing on an Heavy Forwarder

ltrand — Tue, 01 Dec 2015 17:01:27 GMT

Thanks for the clarification, I was under the impression that tcpout was a global setting and as such you could only have one. Selective indexing is a nifty little trick.

Re: Forwardeing and Indexing on an Heavy Forwarder

gcusello — Tue, 29 Sep 2020 08:02:05 GMT

I followed the Docs instructions but I found a problem: logs aren't locally indexed.

I created a new outputs.conf with the described contents,
I inserted in my inputs.conf the described lines:
_INDEX_AND_FORWARD_ROUTING=local in local indexed log stanzas
_TCP_ROUTING=:9997 in remote indexed logs stanzas

but the result is that I don't have local indexing, where cound I search the problem?

Thanks,

Giuseppe

Re: Forwardeing and Indexing on an Heavy Forwarder

gcusello — Tue, 29 Sep 2020 08:02:29 GMT

in addition: if I put "_INDEX_AND_FORWARD_ROUTING=local" in the default stanza of inputs.conf, my HF locally indexes all the logs also the ones I'd like to only send to the remore Indexer.

Re: Forwardeing and Indexing on an Heavy Forwarder

jeffland — Mon, 07 Dec 2015 13:57:05 GMT

Are your settings applied to the proper input stanzas? In the example of the docs, they create a new file monitor. You'll have to apply the settings to your existing inputs.

Re: Forwardeing and Indexing on an Heavy Forwarder

gcusello — Tue, 29 Sep 2020 08:04:46 GMT

yes: I inserted in all input stanzas:

"_INDEX_AND_FORWARD_ROUTING=local" for locally indexed datas
"_TCP_ROUTING=:9997" or remotely indexed datas but the situation is that:
my HF doesn't indexes any datas both local and remote datas
if I put "_INDEX_AND_FORWARD_ROUTING=local" in the default input.conf stanza, my HF indexes all the datas (the locally indexed and the ones to send to the remote indexer) and sends the remote datas to the remote indexer.

Thank you.

Giuseppe

Re: Forwardeing and Indexing on an Heavy Forwarder

jeffland — Mon, 07 Dec 2015 14:17:19 GMT

You shouldn't change things in the default configs, neither generally in the files in the default folder (make your changes in the local folder instead) nor in this case in particular under the [default] stanza in inputs.conf.

Re: Forwardeing and Indexing on an Heavy Forwarder

gcusello — Mon, 07 Dec 2015 14:52:51 GMT

Yes I know it, but I tested many situations to understand why my configuration doesn't work, note that it's the same described in the first example in docs
Thanks.
Giuseppe

Re: Forwardeing and Indexing on an Heavy Forwarder

jeffland — Mon, 07 Dec 2015 18:08:22 GMT

Well, did you put the settings mentioned in the examples under your existing input stanzas? Have you had a look with btool to see if they are applied?

Re: Forwardeing and Indexing on an Heavy Forwarder

mookiie2005 — Thu, 22 Sep 2016 17:59:05 GMT

I downvoted this post because this can be done with route and filtering:
http://docs.splunk.com/documentation/splunk/6.4.3/forwarding/routeandfilterdatad