Getting Data In
Highlighted

How to stop indexing forwarded data from heavy forwarder that indexes locally

Reading from article : Does data indexed and forwarded from a heavy forwarder to indexer would charge twice?

Any indexed forwarded events from a Heavy forwarded are NOT licensed twice.

When Indexing and forwarding from a Heavy Forwarder, the licensing is only used at the Heavy Forwarder, since indexed Data sent to the Indexer, doesn't go through the Parsing queue (as well as the Aggregator and Typing queues).

I have setup the following on my Heavy Forwarder:

outputs.conf:

defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = rdbrsdem03.ref.clp7.local:9997
indexAndForward=true

props.conf

[source::tcp:9999]
BREAK_ONLY_BEFORE=^CEF\:0\|

So on my heavy forwarder, I am sending indexed data to my indexer (rdbrsdem03), and it also filters all events that start with CEF:0|

When I check licensing it seems as if the events ARE being indexed on both the Heavy Forwarder and Indexer.

Can someone provide me with a search possibly using the 'summary' index that proves the events are only being index at the Heavy Forwarder, please?

I have a developer license at the moment so would like to prove that events that need to be indexed at the Heavy Forwarder (due to local users in a remote site being able to search events of their local hardware events) and then not being reindexed (in effect doubling licensing costs) on the Indexer.

Hope this all makes sense, please let me know if there is anything further you may need.

kind regards

Damindra

0 Karma
Highlighted

Re: How to stop indexing forwarded data from heavy forwarder that indexes locally

SplunkTrust
SplunkTrust

Where did you read that index-and-forward does not count twice against your license? I believe that's incorrect, but would like to see your source.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: How to stop indexing forwarded data from heavy forwarder that indexes locally

Hiya, the source of the answer was here on Splunk Answers

https://answers.splunk.com/answers/337523/does-data-indexed-and-forwarded-from-a-heavy-forwa.html

kind regards

Damindra

0 Karma
Highlighted

Re: How to stop indexing forwarded data from heavy forwarder that indexes locally

SplunkTrust
SplunkTrust

Thanks for the citation. That answer has since changed.

Information on Answers is not official and not always definitive. See this answer: https://answers.splunk.com/answers/506909/heavy-forwarder-as-indexer-and-license-usage.html
I'm struggling to find this mentioned in official Splunk docs.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: How to stop indexing forwarded data from heavy forwarder that indexes locally

SplunkTrust
SplunkTrust

We recently had this discussion on the Slack usergroups. A heavy forwarder doing indexing is an *indexer. * License usage gets applied when events get written to disk. This means, when you index twice, your license gets hit twice also.

Skalli

0 Karma
Highlighted

Re: How to stop indexing forwarded data from heavy forwarder that indexes locally

Esteemed Legend

You have no configurations that "filter". The BREAK_ONLY_BEFORE=^CEF\:0\| is a (poorly-performing) LINE_BREAKING configuration. Even so, I am unclear on your goal. Please fill out this chart:

| NODE|  IDX?  |  FWD?  |
+-----+--------+--------+
|  HF | YES/NO | YES/NO |
| IDX | YES/NO |   N/A  |
0 Karma
Highlighted

Re: How to stop indexing forwarded data from heavy forwarder that indexes locally

| NODE| IDX? | FWD? |
2. +-----+--------+--------+
3. | HF | YES/| YES|
4. | IDX | YES/ | N/A |

Hope this makes sense, the reason is there needs to be local searching on the HF.

What would you advise in regards to the LINE_BREAKING?

thanks

0 Karma