Getting Data In

How can I forward only _internal index data from indexer to the new indexer?

benazir
Explorer

I am facing a problem in forwarding the _internal data to the new indexer.

my case is I have to forward only _internal data from all the indexers to new indexer servers because in our environment we have dedicated indexer for _internal data.

when i do this below entry in one of the indexer
inputs.conf:

[monitor:///opt/splunk/idx/splunk/var/log/splunk]
_TCP_ROUTING = management

outputs.conf

[tcpout]
forwardedindex.0.blacklist = .*
forwardedindex.1.whitelist = _internal
forwardedindex.2.whitelist = _audit
forwardedindex.filter.disable = false
disabled=false

[tcpout:management]
server = 10.178.48.66:9997

This makes all the data to forward from this particular indexer to the new indexer, I need only _internal data to get forwarded.

I tried using props.conf and transforms.conf too. It's not working. I don't want to store the _internal data in this indexer, it should present only in the new indexers.
Kindly need your help.

0 Karma

somesoni2
Revered Legend

Try with this outputs.conf (should be etc/apps under some_app/local OR last resort, under etc/system/local)

[tcpout]
indexAndForward = true

[tcpout:management]
server = 10.178.48.66:9997

[indexAndForward]
index=true
0 Karma

benazir
Explorer

I tried this option, what it does it , it keeps a copy of internal logs here in the old indexers and forward to new indexers too.

but my case is , I need to see the _internal data of that particular indexers only in the new indexers, not on the source indexer, when I search data from search head for _internal index..
since we have dedicated search heads , for different cluster of indexers.

Kindly need to your advice, how to just forward, without doing local indexing .

0 Karma

benazir
Explorer

I haven given the outputs.conf file like below :
[tcpout]
forwardedindex.0.blacklist = .*
forwardedindex.1.whitelist = _internal
forwardedindex.2.whitelist = _audit
forwardedindex.filter.disable = false
disabled=false
indexAndForward = true

[tcpout:management]
server = 10.178.48.66:9997

[indexAndForward]
index = true

Now this is how it works, I cant find any other data forwarded to new management indexer ( that's good)
but the problem is _internal data is routed to main index in the new server - 10.178.48.66 and missing few logs like splunkd,metrics all.
meantime in the old indexer I am still seeing the data from main as well as _internal indexes.

0 Karma

deepashri_123
Motivator

Is there any reason why a particular indexer set for internal indexes only? This is not the best practice to do so.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...