Getting Data In

How/where to tell a new data input to use this new index in a cluster?

MikeVenable
Path Finder

I have a cluster environment, 3 indexers and one Master indexer/DMC/LM, a deployment server, syslog-ng Heavy Forwarder, and two search heads. I understand that to make a new index I just update the indexes.conf on the master, and the master will update the index slaves.

  1. If i wanted add a new data input from a a windows box and installed the universal forwarder on the windows box. From my company's old deployment records it says the forwarder points to our deployment server and the deployment server decides what indexer to send the data too for load balancing. When install the universal forwarder I found that there was no where to specify an index that I made in the cluster. Where do I specify this already made index for this new data via the universal forwarder? I know inputs.conf is used in this mater but on what instance to update the inputs.conf?

  2. My second question is if I wanted to add data via the Heavy Forwarder Instance. Now I understand that I tell the Splunk Forwarder instance to look at a directory and pull the files located there. Then the forwarder sends this data to the index clusters. So my question is the same, where do I specify this already made index for this new data?

Thanks for the help

0 Karma
1 Solution

pgerke_cc
Explorer

If i wanted add a new data input from a a windows box and installed the universal forwarder on the windows box. From my company's old deployment records it says the forwarder points to our deployment server and the deployment server decides what indexer to send the data too for load balancing. When install the universal forwarder I found that there was no where to specify an index that I made in the cluster. Where do I specify this already made index for this new data via the universal forwarder? I know inputs.conf is used in this mater but on what instance to update the inputs.conf?

Usually the DS just sends the .conf files to the forwardes and has nothing to do with the load balancing. That is defined in the outputs.conf on the forwarder. Usually the switching for loadbalancing is per time intervalls (guess default here is 30 sec) but can also be changed to data thoroughput.
Using a DS is prefered way to distribute. conf files, espacially for forwarders on windows. As you need to alter the .conf files on the windows admin privileges to edit them.

The index is specified in the inputs.conf on the forwarder in the monitor stanza.

[monitor:\\<path>]
index=<tbd>

View solution in original post

pgerke_cc
Explorer

If i wanted add a new data input from a a windows box and installed the universal forwarder on the windows box. From my company's old deployment records it says the forwarder points to our deployment server and the deployment server decides what indexer to send the data too for load balancing. When install the universal forwarder I found that there was no where to specify an index that I made in the cluster. Where do I specify this already made index for this new data via the universal forwarder? I know inputs.conf is used in this mater but on what instance to update the inputs.conf?

Usually the DS just sends the .conf files to the forwardes and has nothing to do with the load balancing. That is defined in the outputs.conf on the forwarder. Usually the switching for loadbalancing is per time intervalls (guess default here is 30 sec) but can also be changed to data thoroughput.
Using a DS is prefered way to distribute. conf files, espacially for forwarders on windows. As you need to alter the .conf files on the windows admin privileges to edit them.

The index is specified in the inputs.conf on the forwarder in the monitor stanza.

[monitor:\\<path>]
index=<tbd>

MikeVenable
Path Finder

Thanks, this helped a lot.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Thanks for the Memories! Splunk University, .conf25, and our Community

Thank you to everyone in the Splunk Community who joined us for .conf25, which kicked off with our iconic ...

Data Persistence in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. What happens if the OpenTelemetry collector ...

Introducing Splunk 10.0: Smarter, Faster, and More Powerful Than Ever

Now On Demand Whether you're managing complex deployments or looking to future-proof your data ...