How to stop indexing forwarded data from heavy forwarder that indexes locally

[damindragunatil]

Reading from article : Does data indexed and forwarded from a heavy forwarder to indexer would charge twice?

Any indexed forwarded events from a Heavy forwarded are NOT licensed twice.

When Indexing and forwarding from a Heavy Forwarder, the licensing is only used at the Heavy Forwarder, since indexed Data sent to the Indexer, doesn't go through the Parsing queue (as well as the Aggregator and Typing queues).

I have setup the following on my Heavy Forwarder:

outputs.conf:

defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = rdbrsdem03.ref.clp7.local:9997
indexAndForward=true

props.conf

[source::tcp:9999]
BREAK_ONLY_BEFORE=^CEF\:0\|

So on my heavy forwarder, I am sending indexed data to my indexer (rdbrsdem03), and it also filters all events that start with CEF:0|

When I check licensing it seems as if the events ARE being indexed on both the Heavy Forwarder and Indexer.

Can someone provide me with a search possibly using the 'summary' index that proves the events are only being index at the Heavy Forwarder, please?

I have a developer license at the moment so would like to prove that events that need to be indexed at the Heavy Forwarder (due to local users in a remote site being able to search events of their local hardware events) and then not being reindexed (in effect doubling licensing costs) on the Indexer.

Hope this all makes sense, please let me know if there is anything further you may need.

kind regards

Damindra

[damindragunatil]

| NODE| IDX? | FWD? |
2. +-----+--------+--------+
3. | HF | YES/| YES|
4. | IDX | YES/ | N/A |

Hope this makes sense, the reason is there needs to be local searching on the HF.

What would you advise in regards to the LINE_BREAKING?

thanks

[woodcock]

You have no configurations that "filter". The BREAK_ONLY_BEFORE=^CEF\:0\| is a (poorly-performing) LINE_BREAKING configuration. Even so, I am unclear on your goal. Please fill out this chart:

| NODE|  IDX?  |  FWD?  |
+-----+--------+--------+
|  HF | YES/NO | YES/NO |
| IDX | YES/NO |   N/A  |

[skalliger]

We recently had this discussion on the Slack usergroups. A heavy forwarder doing indexing is an *indexer. * License usage gets applied when events get written to disk. This means, when you index twice, your license gets hit twice also.

Skalli

[damindragunatil]

Hiya, the source of the answer was here on Splunk Answers

https://answers.splunk.com/answers/337523/does-data-indexed-and-forwarded-from-a-heavy-forwa.html

kind regards

Damindra

[richgalloway]

Thanks for the citation. That answer has since changed.

Information on Answers is not official and not always definitive. See this answer: https://answers.splunk.com/answers/506909/heavy-forwarder-as-indexer-and-license-usage.html
I'm struggling to find this mentioned in official Splunk docs.

[richgalloway]

Where did you read that index-and-forward does not count twice against your license? I believe that's incorrect, but would like to see your source.