Getting Data In

Heavy Forwarder Costs and Licenses

Contributor

I've already read that I can use a "Free" or "Forwarder" License to implement a Heavy Forwarder.

Is this correct?

Is the forwarded limited to 500 MB throughput when using the Free license?

What does a Forwarder License cost?

1 Solution

Communicator

The only place you pay money for a license is in what you are indexing.
Every other component of Splunk is of no cost.

If you have something like this:

[Universal Forwarder] -> -> (100 MB raw data) -> -> [Heavy Forwarder] -> (50 MB filtered data) -> [Indexer]

A universal forwarder sending a bunch of raw data to a heavy forwarder. That heavy forwarder does some whitelist/blacklists on the data, and cuts it down to around half the size, and then passes it onto the Indexer.

The only data that would count towards your license usage is what reaches the indexer.
Your heavy forwarder could get 1 TB of data, and only forward along 1GB of it. You'll use 1GB of license that day.

View solution in original post

Path Finder

Rob, thanks for your both questions.

Splunk should release a way to keep local _* Indexs out of the indexandforward=false config.

That´s good for local auditing and troubleshooting, but that´s bad on a scenario that the space on a cluster is expensive compared to a heavy Forwarder with lots of gigs free on it´s local disk.

0 Karma

Motivator

@theunf, you can configure how long you keep data written to the _audit and _internal indexes. A shorter retention time will mean less disk space required. As far as clustering, you can turn replication on/off at an index level.

Also keep in mind, in most scenarios, it's best to avoid heavy forwarders altogether when it comes to indexing due to the fact that they will usually complicate your indexing and props rules, and not scale as easily.

0 Karma

Motivator

Summary indexing won't count against your license usage in most cases.
http://answers.splunk.com/answers/8015/how-does-splunk-determine-data-is-being-summarized-and-thus-n...

Report acceleration (aka automagic summary indexing) won't count against your license usage.

Both could require additional storage use.

0 Karma

Motivator

If you set IndexAndForward=False on the Heavy Forwarder, you won't be charged license usage at the Heavy Forwarder, only at your indexer.

If you set IndexAndForward=True on the Heavy Forwarder, you will be charged licensing use on the Heavy Forwarder. Essentially the Heavy Forwarder is acting just like an indexer.

I don't believe you will be charged for a Heavy Forwarder or indexer sending previously indexed events to another downstream Splunk indexer (best to test or contact support to be sure) or a 3rd party system.

You will be charged twice if you have a universal forwarder or heavy forwarder send two replicas of the same yet to be indexed event data to two different indexers or groups of indexers. This is really only for special DR scenarios and I believe Splunk may offer a discounted license for the 2nd copy. Index replication has likely replaced the need to do this in most cases since it now supports site awareness. I've only duplicated events for smaller sets of data where I wanted to have a universal forwarder route events to both a QA and Production environment at the same time which were otherwise independent from each other.

0 Karma

Path Finder

,Rob,

You´ve mentioned right what I´m confused.

1st) a Heavy Forwarder with IndexAndForward=True on outputs.conf forwards logs to an Indexer,
both using the same licensing server. Would it make license count twice for each inputed log ?

2nd) a Search Head on a distributed deployment running summarizations need to forward results to the cluster.
Does the forwarded summarized logs counts on the license of the indexers ?

0 Karma

Motivator

Indexing volume is only one aspect of the license.
Enterprise features is the other.

Typically, all non Universal Forwarder Splunk instances need to communicate with the license server.

A license is definitely required for the following components to take advantage of enterprise features:
Search Head
Indexer
Cluster Master
Deployment Server

A Heavy Forwarder could require communication with the license server depending on whether you told it to index files locally or route events to a non-splunk system and/or used any of the enterprise features such as AD authentication.

See the matrix for enterprise features.
http://www.splunk.com/view/SP-CAAAE8W

About Splunk Free
http://docs.splunk.com/Documentation/Splunk/latest/Admin/MoreaboutSplunkFree#Is_Splunk_Free_for_you....

Communicator

The only place you pay money for a license is in what you are indexing.
Every other component of Splunk is of no cost.

If you have something like this:

[Universal Forwarder] -> -> (100 MB raw data) -> -> [Heavy Forwarder] -> (50 MB filtered data) -> [Indexer]

A universal forwarder sending a bunch of raw data to a heavy forwarder. That heavy forwarder does some whitelist/blacklists on the data, and cuts it down to around half the size, and then passes it onto the Indexer.

The only data that would count towards your license usage is what reaches the indexer.
Your heavy forwarder could get 1 TB of data, and only forward along 1GB of it. You'll use 1GB of license that day.

View solution in original post

Path Finder

Really helpful answer. Thank you very much.

0 Karma

Motivator

Do i need make the heavy forwarder as a slave , in Master license server?

Explorer

In short, you may want to do exactly that -- point to your lic. server. This depends on the "Features" needed. The following link (based on post by rob_jordan) lists 'exempt features' you will not have access to, without an enterprise license. Use this to help determine how it may apply to your environment. http://docs.splunk.com/Documentation/Splunk/latest/Admin/MoreaboutSplunkFree#What_is_included_with_S...
Note: I realize this question is dated but continues to be a relevant question.

0 Karma

Motivator

am checking the "solved" box for this answer for you @froth. Don't forget to do that when your questions are answered, that greatly helps the visibility and quality of content on the site. Thanks!

Contributor

Great answer! Very helpful, Thanks

0 Karma