Getting Data In

Cannot forward one specific index between indexers

lenrigodoy
Explorer

I'm working on an indexer to try to forward all data ingested with IT Essentials Work + Splunk Add-on for Unix & Linux to a remote indexer cluster. Until now, that indexer is receiving events into all itsi_* indexes, but, when I try to setup the forwarding option into that indexer, I cannot set the forwardedindex.n.whitelist and blacklist to forward only the itsi_* indexes to the IDX Cluster. I've try to overwrite all default whitelists and blacklists on local and reset whitelists with itsi_* indexes, but, this still forwarding all indexes, nor only itsi_* indexes.

My outputs.conf file is like following:

[tcpout]
defaultGroup = default-autolb-group
forwardedindex.0.whitelist =
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = (itsi_grouped_alerts|itsi_im_meta|itsi_im_metrics|itsi_import_objects|itsi_notable_archive|itsi_notable_audit|itsi_summary|itsi_summary_metrics|itsi_tracked_alerts)
indexAndForward = 1

[tcpout:default-autolb-group]
disabled = false
server = HFtoIDXCluster:9997
useACK = true

If I use a "default" config option, overwriting the lists not resetting (not declaring the default 3 lists empty on the tcpout stanza) I have the same behaviour. This is the first time I try to set forwarding options from an indexer. I need to forward this data because it's used for administration of each Splunk instances, and it's required to get into a specific Splunk Enterprise cluster, but, all other indexes it's not required to be forwarded. Have I miss something to specify into config files?

Best regards

Labels (1)
Tags (2)
0 Karma

tshah-splunk
Splunk Employee
Splunk Employee

Hey @lenrigodoy,

Did you try specifying the outputs in the below manner?

[tcpout]
defaultGroup = default-autolb-group
forwardedindex.0.whitelist =(itsi_grouped_alerts|itsi_im_meta|itsi_im_metrics|itsi_import_objects|itsi_notable_archive|itsi_notable_audit|itsi_summary|itsi_summary_metrics|itsi_tracked_alerts)
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
indexAndForward = 1

The filters will be working in the sequential manner of the integers provided after the forwardedindex parameter. And in your case, I see there are 2 parameters with forwardedindex.0.whitelist in the outputs.conf. And sequentially, the filter you need is present in the parameter present at the bottom. 

---
If you find the answer helpful, an upvote/karma is appreciated
0 Karma

lenrigodoy
Explorer

I've follow the Docs about route data

https://docs.splunk.com/Documentation/Splunk/8.2.4/Forwarding/Routeandfilterdatad#Filter_data_by_tar...

In this doc, it's recommended to do:

"If you want to forward only the data targeted for a single index (for example, as specified in inputs.conf), and drop any data that is not a target for that index, configure outputs.conf in this way:

[tcpout]
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =

#Forward data for the "myindex" index
forwardedindex.0.whitelist = myindex

This first disables all filters from the default outputs.conf file. It then sets the filter for your own index. Be sure to start the filter numbering with 0: forwardedindex.0."

Now, I'm testing your config, I'll update my answer in case of that config works. Otherwise, I will test other configs to find the working one.

Tags (2)
0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...