topic Re: Missing Source data from Forwarder in Getting Data In

Missing Source data from Forwarder

thomas_bengtsen — Fri, 28 Oct 2011 13:15:12 GMT

Hi Group,

I am new to the Splunk thing so bear with me.
I have installed an indexer, configured it to look at some local log files and that seems to work ok. I have also installed a forwarder on another machine and configure it to monitor a file and connect to the Indexer. As far as I can tell the file is being monitored and the data is sent to the Indexer and being indexed – at least I can see the index having count and size_bytes if I look under “Status –> Index activity -> Index activity overview”.

The problem is that if I look on the search page I can only see one source – namely the local file. My searches do not show any entries form the file on the indexer. Additionally – and this I find very strange – if under “Status –> Index activity -> Index activity overview” I drill down into the index for the remote server it shows me the entries from the local file on the indexer.
(Splunk 4.2 on Solaris 10 X86)

Thanks

Re: Missing Source data from Forwarder

kristian_kolb — Fri, 28 Oct 2011 14:05:45 GMT

If you could share a little more informatiion, it would be easier to help you.

What do the inputs.conf and outputs.conf look like on the forwarder.

What do the inputs.conf and indexes.conf look like on the indexer.

/kristian

Re: Missing Source data from Forwarder

thomas_bengtsen — Mon, 28 Sep 2020 10:02:10 GMT

Forwarder:

outputs.conf
[tcpout]
defaultGroup = indexer1
heartbeatFrequency=10

Defect 396475

maxQueueSize=10000

[tcpout:indexer1]
server=192.168.53.6:9997

inputs.conf
host = TEST01A
[splunktcp://9997]
[monitor:///com/logs/grp/app2.log]
disabled = false
followTail = 0
index = app2

splunk list monitor -auth admin:*****

Monitored Directories:
$SPLUNK_HOME/var/log/splunk
...lines removed
Monitored Files:
$SPLUNK_HOME/etc/splunk.version
$SPLUNK_HOME/var/log/splunk/splunkd.log
$SPLUNK_HOME/var/spool/splunk
/com/logs/grp/app2.log

Re: Missing Source data from Forwarder

thomas_bengtsen — Mon, 28 Sep 2020 10:02:13 GMT

Indexer:

No outputs.conf

inputs.conf
[splunktcp://9997]

[monitor:///com/logs/grp/app1.log]
disabled = false
followTail = 0
index = app1

indexes.conf
[app2]
coldPath = $SPLUNK_DB/app2/colddb
homePath = $SPLUNK_DB/app2/db
thawedPath = $SPLUNK_DB/app2/thaweddb
disabled = 0

[app1]
coldPath = $SPLUNK_DB/app1/colddb
homePath = $SPLUNK_DB/app1/db
thawedPath = $SPLUNK_DB/app1/thaweddb

Re: Missing Source data from Forwarder

thomas_bengtsen — Fri, 28 Oct 2011 15:14:45 GMT

Formatting seems screwed up – sorry about that.
Thanks

Re: Missing Source data from Forwarder

alacercogitatus — Fri, 28 Oct 2011 15:35:30 GMT

The first thing I would to is remove [splunktcp://9997] from the forwarder inputs.conf.
The next thing I would to is add the [app2] index stanza to the system/local/indexes.conf on the forwarder. Then the forwarder knows it exists as a valid index, and can forward it. I use something similar on one of my systems and I have the index declared on both systems, and it works fine. It may or may not use that index on the forwarder, it depends on the type of forwarder.

Re: Missing Source data from Forwarder

thomas_bengtsen — Fri, 28 Oct 2011 16:16:56 GMT

Thanks for the suggestion but still no change.

I think it it something simpler - remember I am new to this.

When I check the Summery screen the only Source that show up is the source I added via the GUI on the Indexer itself - app1.

There are other indexes apart from app1 and app2 from splunks own files and they don't show up either. It seems to me just a question of adding this data which already exists in indexes to the searchable view.

Thanks

Re: Missing Source data from Forwarder

alacercogitatus — Fri, 28 Oct 2011 16:57:13 GMT

Is splunk indexer listening on the port? netstat -an | grep LISTEN should return *.9997

Re: Missing Source data from Forwarder

kristian_kolb — Fri, 28 Oct 2011 19:46:32 GMT

Well, there may be the simplest of possible explanations.

Have you enabled the correct indexes to be searched by default in Manager > Access Controls > Roles > .

If you're really new to this, you are probably logging in as 'admin', and you should check if your newly created indexes for app1 and app2 are enabled for default searching. (almost at the bottom of the page.)

Unless an index is selected for default searching, your searches will not return any events from it unless you include index=<your_index> as part of your search.

hope this helps,

/Kristian

And no, you do not need to define the index in indexes.conf on the forwarder

Re: Missing Source data from Forwarder

thomas_bengtsen — Tue, 01 Nov 2011 22:06:31 GMT

Yes - I am really that new to this 🙂

Thanks for your help

Re: Missing Source data from Forwarder

ehoward — Wed, 11 Jan 2012 15:49:19 GMT

I am also have a problem with Indexing Volume stats. I am logged in as Admin with full permissions and role access to everything, yet when run Status, Indexing Volume I only see data for is for _thefishbucket and _internal. I have a lot of Windows WMI event data being forwarded to the Indexer the indexing stats are not showing when when I display the Indexing Volume by Source/Sourcetype.