topic Re: Does Splunk reindex data from another forwarder? in Getting Data In

Does Splunk reindex data from another forwarder?

Jamshed — Thu, 01 Aug 2013 18:46:53 GMT

I have a full version of Splunk Indexer running on one machine. It is indexing data and sending the index data to another instance of Splunk Indexer running on another machine. Does the receiving instance of Splunk reindex that data?
Is this received (and indexed) data counted in the 500 mb?

Thanks.

Re: Does Splunk reindex data from another forwarder?

mloven_splunk — Thu, 01 Aug 2013 18:49:29 GMT

How are you sending the data from the first Splunk server to the second?

Re: Does Splunk reindex data from another forwarder?

Jamshed — Thu, 01 Aug 2013 18:51:59 GMT

I am using Splunk's forwarder capability. I am NOT using a universal forwarder.

Re: Does Splunk reindex data from another forwarder?

mloven_splunk — Thu, 01 Aug 2013 19:00:31 GMT

So, does that mean you converted the first server to a lightweight or heavy forwarder?

Re: Does Splunk reindex data from another forwarder?

Jamshed — Thu, 01 Aug 2013 19:07:03 GMT

I'm not entirely sure. I think it is considered a heavy forwarder because I still have Splunk Web enabled and all.

My question is, is the data counted for each instance of Splunk? Or does the data index on the first server not count as indexing data for the second server?

Re: Does Splunk reindex data from another forwarder?

mloven_splunk — Thu, 01 Aug 2013 19:11:43 GMT

More info about heavy vs. lightweight forwarders here.

I understand your question, I'm just trying to determine if you're actually indexing on both servers. If you are, you probably don't need to be.

And yes, if you're indexing on both servers, both count against the license. I'm assuming that you've got the free 500MB license on each? If so, then both server's licenses would be getting dinged.

Re: Does Splunk reindex data from another forwarder?

Jamshed — Thu, 01 Aug 2013 19:14:02 GMT

Exactly. My next question is, how can I only have it index on the first server, send the index data to the second, and get dashboards etc from the second without indexing it?

Does the second instance of Splunk HAVE to index the data in order to get anything meaningful out of the data (alerts, etc?

Thanks!

Re: Does Splunk reindex data from another forwarder?

mloven_splunk — Thu, 01 Aug 2013 19:35:04 GMT

You have two options here:

Turn the first server into a lightweight or heavy forwarder per the instructions in the link I posted before.
Turn the second server into a search head, and the first server into a search peer.

The second option isn't really an option for you if you're on the free license.

I'd go with the first option. When you do, the data will no longer be getting indexed on the first server. The first server will just forward the data to the second server, where it will be indexed, and can be searched upon.

For reference, here is some info on the capabilities for the different types of licenses.

Re: Does Splunk reindex data from another forwarder?

Jamshed — Thu, 01 Aug 2013 19:39:26 GMT

Great, thank you for the info.

Let's say that now, instead of that one first forwarder, I have 10. 10 'heavy forwarders' pointing one free license will go over the 500MB limit.

I am trying to index the data locally, send that data to a central server for viewing. This way each instance stays below 500MB.

Does this seem like a reasonable architecture?

Or does that central server have to re-index all that data, in which case there is no point in indexing on the local servers at the first level.

Thank you for your time.

Re: Does Splunk reindex data from another forwarder?

mloven_splunk — Thu, 01 Aug 2013 19:49:45 GMT

This doesn't seem like a reasonable architecture. What you're describing isn't a forwarder (heavy, or otherwise), but an indexer. And distributed search isn't available with the free license.

Your only options here are either to be more choosy about what you forward, so you can stay below the 500MB limit, or pony up for an Enterprise license.

Re: Does Splunk reindex data from another forwarder?

Jamshed — Thu, 01 Aug 2013 19:52:19 GMT

Ok. But my initial question remains - if one instance of Splunk indexes data, sends that indexed data to another instance of Splunk, does that instance have to re-index that data to get anything meaningful out of it (such as alerts, etc)?

Re: Does Splunk reindex data from another forwarder?

mloven_splunk — Thu, 01 Aug 2013 19:56:47 GMT

With the free license, yes.

Re: Does Splunk reindex data from another forwarder?

Jamshed — Thu, 01 Aug 2013 19:58:21 GMT

Thank you.

Re: Does Splunk reindex data from another forwarder?

mloven_splunk — Thu, 01 Aug 2013 20:16:24 GMT

No problem. If I've answered your question, please remember to accept one of the responses so the question shows as answered.

Thanks!