Forwarding data into indexer

wu_weidong — Tue, 28 Sep 2021 10:38:47 GMT

Hi,

I have set up a Splunk Enterprise instance (version 8.2.1) and a Universal Forwarder instance on Docker on the same machine, and I'm trying to forward data into the Splunk indexer. Here's what I have so far:

On the Splunk Enterprise instance (1.1.1.1):

Created an app named "abc"
Created an index named "abc_idx" on app "abc"
Created a sourcetype named "abc_data" on app "abc"

On the Splunk forwarder:

Added the indexer: "./bin/splunk add forward-server 1.1.1.1:9997"
My very next command was "./bin/splunk add monitor /splunk_forward/log"
Then I realized I wanted the monitored logs to be added to the index "abc_idx" and using the sourcetype "abc_data", so I removed the monitor, and then restarted the container.

This is when I see the events appearing in the "main" index, so I believe the files did get forwarded.

I then ran on the Splunk forwarder:

./bin/splunk add monitor /splunk_forward/log -index abc_idx -sourcetype abc_data

But I did not see any event on the index "abc_idx".

However, if I run the "oneshot" command, the events show up in the index "abc_idx"

Is Splunk refusing to (re)index the same files again, even though they are going to different indexes?

Also, I thought the commands I typed would end up in "/opt/splunkforwarder/etc/system/local/inputs.conf"? But I only see "[splunktcp://9997]" in it, not the folder I'm monitoring. Am I looking at the wrong file?

However, I see the following in "/opt/splunkforwarder/etc/system/local/outputs.conf":

[tcpout:default-autolb-group]

server = 1.1.1.1:9997

[tcpout-server://1.1.1.1:9997]

So why did my indexer configuration become part of the config file? Preferably, I would like to configure the forwarder using the config files, but I'm not sure exactly which ones to modify - local/inputs.conf and anything else?

Thank you.

Re: Forwarding data into indexer

Stefanie — Fri, 01 Oct 2021 18:14:11 GMT

Is Splunk refusing to (re)index the same files again, even though they are going to different indexes?

Yes that is correct. Splunk will not re-index data that has already been indexed.

Also, I thought the commands I typed would end up in "/opt/splunkforwarder/etc/system/local/inputs.conf"? But I only see "[splunktcp://9997]" in it, not the folder I'm monitoring. Am I looking at the wrong file?

Thats where I would expect it to be. You could check for /opt/splunkforwarder/etc/system/default/inputs.conf

So why did my indexer configuration become part of the config file? Preferably, I would like to configure the forwarder using the config files, but I'm not sure exactly which ones to modify - local/inputs.conf and anything else?

Your indexer configuration became part of the config file because all configurations within Splunk are stored as conf files.

If you would like to configure forwarding using the config files then yes you can modify the local/inputs.conf. You may have to restart the forwarder depending on what you add to it.

However, when we have to add new inputs we will create an app which contains a new inputs.conf. Then that app can be managed and updated without having to touch the forwarder. 🙂

Hope this helps!

topic Forwarding data into indexer in Getting Data In

Forwarding data into indexer

Re: Forwarding data into indexer