topic Re: Configure input.txt Universal forwarder in Getting Data In

Configure input.txt Universal forwarder

sieutruc — Mon, 28 Sep 2020 12:22:14 GMT

Hello,

I create an idexer server as server01 and a Universal forwarder that monitors and forwards a file in real time to indexer "example" of the indexer with the configuration below:
C:\Program Files\SplunkUniversalForwarder\etc\system\local\input.conf

[monitor://D:\AFCSystem\log\log_file.txt]
disabled = 0
_TCP_ROUTING = splunkserver01_9997
index = example
followTail = 1

and output.conf at the same directory

[tcpout]
defaultGroup = splunkserver01_9997

[tcpout:splunkserver01_9997]
disabled = false
server = splunkserver01:9997

[tcpout-server://splunkserver01:9997]

But when i view it in "example" indexer, i didn't see anything ?
Do you know the problem ?

Re: Configure input.txt Universal forwarder

kristian_kolb — Mon, 28 Sep 2020 12:22:16 GMT

A couple of things,

If you call the file inputs.txt you will have problems, it should be called inputs.conf, otherwise Splunk will not recognize it. The same goes for outputs.conf

The order of precedence can be found in the docs.

You must configure the indexer to listen on port 9997. (manager -> forwarding and receiving -> configure receiving -> new).

Make sure that you have no firewalls in between (or local), blocking the traffic.

By the way, I don't think you'll need the _TCP_ROUTING stuff in the form you've written it;

(from the docs on inputs.conf)

_TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>,<tcpout_group_name>, ...
* Comma-separated list of tcpout group names.
* Using this, you can selectively forward the data to specific indexer(s).
* Specify the tcpout group the forwarder should use when forwarding the data.
  The tcpout group names are defined in outputs.conf with [tcpout:<tcpout_group_name>].
* Defaults to groups specified in "defaultGroup" in [tcpout] stanza in outputs.conf.
* To forward data from the "_internal" index, _TCP_ROUTING must explicitly be set to either "*" or
  a specific splunktcp target group.

UPDATE:

In order to do that, you'll need to clean out the fishbucket on the forwarding side. The fishbucket is where splunk keeps track of which files/events it has already read/forwarded.

http://splunk-base.splunk.com/answers/2954/how-can-i-re-index-all-the-data-in-my-environment

http://splunk-base.splunk.com/answers/2834/light-forwarder-syslog-fishbucket-problem

http://splunk-base.splunk.com/answers/46780/reset-splunkforwarder-to-re-read-file-from-beginning

Just be aware that if you do clean out the fishbucket on the forwarder, you'll get get some duplicate events, e.g. your file has events A, B, C and D, but your index contains only C and D (and you want to index A and B as well.) If you clean the fishbucket, you'll have A,B,C,C,D,D in your index. Depending on what you have in your index, you may also want to clean out the index on the indexer as well, giving you A,B,C,D in the index.

Hope this helps,

Kristian

Re: Configure input.txt Universal forwarder

sieutruc — Mon, 03 Sep 2012 12:07:43 GMT

Sorry for my wrong typing, all files have .conf extention, and all firewalls are turned off. But i still don't see it in example indexer. Is there a test for that ?

Re: Configure input.txt Universal forwarder

kristian_kolb — Mon, 03 Sep 2012 12:18:04 GMT

Is your indexer listening on port 9997 as per the instructions above?

Re: Configure input.txt Universal forwarder

sieutruc — Mon, 03 Sep 2012 13:40:07 GMT

It works now, but it just recorded in real time all events in the end of the file without the total file. How to catch up all the events in the beginning of the file ? I delete the option the followTail but nothing changes

Re: Configure input.txt Universal forwarder

kristian_kolb — Mon, 03 Sep 2012 13:52:15 GMT

see update above.