How to set the indexes for a single HTTP event collector input in an indexer cluster?

[szabados]

I'm configuring what I believe is the first scenario here: http://dev.splunk.com/view/event-collector/SP-CAAAE73#scen1

I'm stuck at setting the indexes for this input. As I'm running an indexer cluster, my indexers are not visible on the forwarder, so I can't add the index I want to use to store the data coming from the HTTP Event Collector. I see only indexes which are defined on the forwarder itself, like main, summary, etc.

How can I set an index for this?
The other thing, the "Output Group" contains only the "None" value. However, I have outputs configured on this instance, since it is forwarding other events to my cluster. Will be these output settings applied to this by default, or do I have to configure them explicitly?

[santiagn]

any update on solution?

[szabados]

yes: first I create the input on the web gui as described in the documentation, and then I go to the HFWD instance via the filesystem (rdp/ssh), /etc/apps/splunk_httpin (not 100% if this is the apps name, something similar)/local/inputs.conf

In this file, you change the index= to the index you want to use.

hope this helps

[santiagn]

thanks for the response, so for first step i leave outputgroup as none, second step of setup where it asks which index do you want to point to, what did you select? i still dont see my indexes, do i just leave it as main?

[szabados]

Yes, I've left it as main, and then changed it to my "real" index in the *.conf file.

[santiagn]

thank you i will try this

[gblock_splunk]

If you have a single HEC instance forwarding to multiple indexers, then as Jeremiah said, you can create the indexes on the instance to allow you to select them via the Splunk UI. Or you can use our CLI / REST API which do not require the indexes be present. One caveat to this is if you specify the index in the payload (like using the "index" filed), those indexes must be present on the HEC instance. Regardless, the events will not be written locally to these indexes.

As far as forwarding, if you configure default output groups on the box, HEC will use them. The "Output Group" setting is to allow you to configure a group specifically for forwarding HEC events. It will not for example forward _internal or _introspection events.

[Jeremiah]

If you would like to use the UI to set the index, you'll probably need to load an indexes.conf on the heavy forwarder that matches what you load on your indexers. Then, the forwarder will have the same list of indexes for you to choose from. Defining the indexes on the forwarder is fine, if you've configured your heavy forwarder correctly it will not write any data to the local indexes, just create their directory structure within the $SPLUNK_HOME/var/lib/splunk directory. Otherwise, you could define your settings directly in the inputs.conf file for the HEC and bypass the UI.

The forwarder will use your default output settings, even if the Output Group contains "None". You don't have to explicitly configure an additional output group unless you want the ability to route different data to different endpoints.