topic Re: Specific index forwarding to external index tier in Getting Data In

Specific index forwarding to external index tier

sonicZ — Mon, 28 Sep 2020 12:12:22 GMT

Hello,

We have a requirement that certain indexes(SSO and SSO_Summary for this example) in our index cluster send to another external offsite network's splunk forwarder/indexer environment. Can we forward just the "SSO and SSO_Summary" from our index cluster level to this other offsite splunk environment?

We have specific silos that each Splunk agent / intermediate forwarder
cannot talk to other silos intermediate forwarder tiers or end points agents. Because we have these restricted silos, we cannot forward data as easily as some of the standard Splunk examples.
(like the data cloning examples)

So an example silo has Splunk data routing as follows
Front end silo: end points(many) -> Intermediate forwarder(pair) ->
Back end silo: end points(many) -> Intermediate forwarder(pair) ->
Shared silo -> index cluster(4 indexers all Front end, back end data forwards to here)

The only examples i see is forwarding ALL index content or data routing based on events content using regular expressions, how would i just forward specific indexes?

We would have a VPN in place to do this, just checking if this is possible
We are trying to avoid forwarding from the end point agents to the external Splunk environment due to security considerations.

Thanks

Re: Specific index forwarding to external index tier

sonicZ — Mon, 28 Sep 2020 12:12:49 GMT

I didnt realize but looks like we can filter and forward from indexer to indexer now using forwardedindex whitelist blacklist
http://docs.splunk.com/Documentation/Splunk/4.3.3/Deploy/Routeandfilterdatad#Filter_data_by_target_index

So something like this might work?

[tcpout]
defaultGroup = indexer_external
disabled = false

[tcpout:indexer_external]
indexAndForward = true
disabled = false
server = indexer_ip:9997
forwardedindex.filter.disable = false
forwardedindex.2.whitelist = externalIndexName

Re: Specific index forwarding to external index tier

sonicZ — Fri, 02 Nov 2012 23:20:16 GMT

I didnt realize but looks like we can filter and forward from indexer to indexer now using forwardedindex whitelist blacklist http://docs.splunk.com/Documentation/Splunk/4.3.3/Deploy/Routeandfilterdatad#Filter_data_by_target_index

So something like this would work

> [tcpout] defaultGroup =
> indexer_external  disabled = false
> 
> [tcpout:indexer_external]
> indexAndForward = true  disabled =
> false  server = indexer_ip:9997 
> forwardedindex.filter.disable = false 
> forwardedindex.2.whitelist = externalIndexName

Re: Specific index forwarding to external index tier

sonicZ — Fri, 02 Nov 2012 23:20:35 GMT

Splunk support confirmed this.

Re: Specific index forwarding to external index tier

dm1 — Wed, 18 Aug 2021 23:49:50 GMT

Doesn't the forwardedindex filter only works for global [tcpout] stanza as per outputs.conf ?

How did you manage to make it work for targetgroup specific stanza ?