Getting Data In

Specific index forwarding to external index tier

sonicZ
Contributor

Hello,

We have a requirement that certain indexes(SSO and SSO_Summary for this example) in our index cluster send to another external offsite network's splunk forwarder/indexer environment. Can we forward just the "SSO and SSO_Summary" from our index cluster level to this other offsite splunk environment?

We have specific silos that each Splunk agent / intermediate forwarder
cannot talk to other silos intermediate forwarder tiers or end points agents. Because we have these restricted silos, we cannot forward data as easily as some of the standard Splunk examples.
(like the data cloning examples)

So an example silo has Splunk data routing as follows
Front end silo: end points(many) -> Intermediate forwarder(pair) ->
Back end silo: end points(many) -> Intermediate forwarder(pair) ->
Shared silo -> index cluster(4 indexers all Front end, back end data forwards to here)

The only examples i see is forwarding ALL index content or data routing based on events content using regular expressions, how would i just forward specific indexes?

We would have a VPN in place to do this, just checking if this is possible
We are trying to avoid forwarding from the end point agents to the external Splunk environment due to security considerations.

Thanks

Tags (2)
0 Karma
1 Solution

sonicZ
Contributor

I didnt realize but looks like we can filter and forward from indexer to indexer now using forwardedindex whitelist blacklist http://docs.splunk.com/Documentation/Splunk/4.3.3/Deploy/Routeandfilterdatad#Filter_data_by_target_i...

So something like this would work

> [tcpout] defaultGroup =
> indexer_external  disabled = false
> 
> [tcpout:indexer_external]
> indexAndForward = true  disabled =
> false  server = indexer_ip:9997 
> forwardedindex.filter.disable = false 
> forwardedindex.2.whitelist = externalIndexName

View solution in original post

sonicZ
Contributor

I didnt realize but looks like we can filter and forward from indexer to indexer now using forwardedindex whitelist blacklist http://docs.splunk.com/Documentation/Splunk/4.3.3/Deploy/Routeandfilterdatad#Filter_data_by_target_i...

So something like this would work

> [tcpout] defaultGroup =
> indexer_external  disabled = false
> 
> [tcpout:indexer_external]
> indexAndForward = true  disabled =
> false  server = indexer_ip:9997 
> forwardedindex.filter.disable = false 
> forwardedindex.2.whitelist = externalIndexName

dm1
Contributor

Doesn't the forwardedindex filter only works for global [tcpout] stanza as per outputs.conf ?

How did you manage to make it work for targetgroup specific stanza ?

0 Karma

sonicZ
Contributor

Splunk support confirmed this.

0 Karma

sonicZ
Contributor

I didnt realize but looks like we can filter and forward from indexer to indexer now using forwardedindex whitelist blacklist
http://docs.splunk.com/Documentation/Splunk/4.3.3/Deploy/Routeandfilterdatad#Filter_data_by_target_i...

So something like this might work?

[tcpout]
defaultGroup = indexer_external
disabled = false

[tcpout:indexer_external]
indexAndForward = true
disabled = false
server = indexer_ip:9997
forwardedindex.filter.disable = false
forwardedindex.2.whitelist = externalIndexName

0 Karma
Get Updates on the Splunk Community!

This Week's Community Digest - Splunk Community Happenings [9.26.22]

Get the latest news and updates from the Splunk Community here! Upcoming User Group Events! 👏 Check ...

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...