Getting Data In

Forwardeing and Indexing on an Heavy Forwarder

gcusello
SplunkTrust
SplunkTrust

Hi at all,
I have a Splunk instance indexing some logs.
I'd like to continue to use the server for its old job but, at the same time, to use the same server (the same Splunk instance or a different one) to forward another log flow to a different Indexer without local indexing.
In other words: I have to locally index some flows and forward to another Indexer a different one.
I know that I can configure outputs.conf to forward logs to different indexers, but is it possible to send a flow to another Indexer and locally index other flows?
Can I do this with a single Splunk instance or do I have to install another Splunk instance (Universal or Heavy) to forward?
Thank you in advance.
Bye.
Giuseppe

0 Karma
1 Solution

ltrand
Contributor

You'll need a new instance as you can only have one tcpout stanza per instance and that stanza is where you set indexAndForward. So yes, but you have to use two Splunk instances. If you can provide more detail on the situation then better guidance can be given, but I would suggest to use the UF to forward off the log that you don't want to index locally but want to forward remotely.

View solution in original post

0 Karma

jeffland
Champion

Your single splunk instance can do both at the same time, see here for docs.

Basically, you can set up any forwarding and/or indexing settings for each input stanza independently.

ltrand
Contributor

You'll need a new instance as you can only have one tcpout stanza per instance and that stanza is where you set indexAndForward. So yes, but you have to use two Splunk instances. If you can provide more detail on the situation then better guidance can be given, but I would suggest to use the UF to forward off the log that you don't want to index locally but want to forward remotely.

View solution in original post

0 Karma

mookiie2005
Communicator

I downvoted this post because this can be done with route and filtering:
http://docs.splunk.com/documentation/splunk/6.4.3/forwarding/routeandfilterdatad

0 Karma

gcusello
SplunkTrust
SplunkTrust

I followed the Docs instructions but I found a problem: logs aren't locally indexed.

  • I created a new outputs.conf with the described contents,
  • I inserted in my inputs.conf the described lines:

  • _INDEX_AND_FORWARD_ROUTING=local in local indexed log stanzas

  • _TCP_ROUTING=:9997 in remote indexed logs stanzas

but the result is that I don't have local indexing, where cound I search the problem?

Thanks,

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

in addition: if I put "_INDEX_AND_FORWARD_ROUTING=local" in the default stanza of inputs.conf, my HF locally indexes all the logs also the ones I'd like to only send to the remore Indexer.

0 Karma

jeffland
Champion

Are your settings applied to the proper input stanzas? In the example of the docs, they create a new file monitor. You'll have to apply the settings to your existing inputs.

0 Karma

gcusello
SplunkTrust
SplunkTrust

yes: I inserted in all input stanzas:

  • "_INDEX_AND_FORWARD_ROUTING=local" for locally indexed datas
  • "_TCP_ROUTING=:9997" or remotely indexed datas but the situation is that:
  • my HF doesn't indexes any datas both local and remote datas
  • if I put "_INDEX_AND_FORWARD_ROUTING=local" in the default input.conf stanza, my HF indexes all the datas (the locally indexed and the ones to send to the remote indexer) and sends the remote datas to the remote indexer.

Thank you.

Giuseppe

0 Karma

jeffland
Champion

You shouldn't change things in the default configs, neither generally in the files in the default folder (make your changes in the local folder instead) nor in this case in particular under the [default] stanza in inputs.conf.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Yes I know it, but I tested many situations to understand why my configuration doesn't work, note that it's the same described in the first example in docs
Thanks.
Giuseppe

0 Karma

jeffland
Champion

Well, did you put the settings mentioned in the examples under your existing input stanzas? Have you had a look with btool to see if they are applied?

0 Karma

jeffland
Champion

No. You can have more than one tcpout stanza, and you can assign these to different inputs. Also, indexAndForward is not the only setting that governs indexing and forwarding; you can route any input any way you like, its called selective indexing. See these docs for more.

ltrand
Contributor

Thanks for the clarification, I was under the impression that tcpout was a global setting and as such you could only have one. Selective indexing is a nifty little trick.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!