topic Re: Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System"" in Getting Data In

Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System""

stuntman2625 — Thu, 05 Nov 2015 18:45:32 GMT

I'm receiving the following message on my Splunk Indexer:

Received event for unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System" host="host::xxx" sourcetype="sourcetype::WinEventLog:System". So far received events from 1 missing index(es).

I've seen the same question posted and resolved in many forums by simply adding the wineventlog index since it doesn't exist by default. However, that does not seem to work for me and I'm sure I'm missing something obvious.

My forwarder is forwarding custom logs, it just will not forward Windows event logs because of the above error. This is what I have in C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\local\inputs.conf:

[WinEventLog://Security]
disabled = 0

[monitor://C:\Program Files (x86)\Entrust\VerificationServer\logs\webservices.log]
disabled = false

If I modify it to explicitly use the main index as below, the event logs come through without any issues:

[WinEventLog://Security]
disabled = 0
index=main

[monitor://C:\Program Files (x86)\Entrust\VerificationServer\logs\webservices.log]
disabled = false

In both cases, my monitored log (webservices.log) gets forwarded successfully.

Using the GUI, I created a Search & Reporting index called wineventlog, restarted both the indexer and forwarder, but nothing comes through. It set the contents of my etc/apps/search/local/indexes.conf file to the following:

[wineventlog]
coldPath = $SPLUNK_DB/wineventlog/colddb
homePath = $SPLUNK_DB/wineventlog/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/wineventlog/thaweddb

I've also tried selecting Distributed Management Console, Home and App Browser under the "App" type when creating the index instead of Search & Reporting, but they all have the same behaviour.

My question is, is there anything else I need to do in order to get my indexer to use this index?

Re: Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System""

lguinn2 — Thu, 05 Nov 2015 19:49:33 GMT

Your forwarder probably IS forwarding these logs - but there is no way for the forwarder to know whether or not the data was successfully indexed.

And it looks like you created the correct index (should be on all your indexers if you have more than one). So that should be fine as well.

However - once the forwarder has sent some data, it will not send the same data again.

If this is your problem, you will need to reset the forwarder's file pointers. You can reset all of them by deleting the fishbucket directory, $SPLUNK_HOME\var\lib\splunk\fishbucket on the forwarder. Or you can use btprobe to reset individual file pointers. Here is an answer that may help (even though it is older): btprobe and re-indexing data

Re: Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System""

stuntman2625 — Thu, 05 Nov 2015 19:58:54 GMT

That may be the case for old data, but if I'm creating new events shouldn't it be sending those going forward? I can produce new event logs by logging in and out of the server, but they still don't show up on the indexer unless I point to index main. I just tried deleting the fishbucket directory as you mentioned and restarting the forwarder, but the behaviour doesn't change.

Thanks!

Re: Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System""

lguinn2 — Thu, 05 Nov 2015 20:36:10 GMT

Ah, my guess is that your user role doesn't search the wineventlog log index by default. Did you try searchingindex=weblog specifically?

Re: Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System""

stuntman2625 — Thu, 05 Nov 2015 20:46:57 GMT

Well now I feel like an idiot, that was the problem! First time setting up Splunk and I've been doing everything as admin so I just assumed it was searching all indexes. Thanks for the quick response!

Re: Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System""

lguinn2 — Thu, 05 Nov 2015 21:50:09 GMT

Don't feel bad! I have done this more than once. Now, I always change the admin role so that it set to search "all non-internal indexes" by default. It's part of the steps I go through whenever I install or update Splunk...

Re: Why is my indexer reporting "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System""

sylim_splunk — Tue, 29 Sep 2020 10:05:50 GMT

If the index exists on your indexers and If it is happening from clustered indexers, has worked fine until the message was found, just all of a sudden it complains about the error after an activity on the cluster.
Check if the index gets disabled by splunk instance. If then this must be the cause of the message. You may need to check if you have any bucket id conflicts like below which is caused by bucket replication ;

ERROR IndexerService - Error intializing IndexerService: idx=MyIndex bid=MyIndex~25~9B9D1F9-8EA5-4C73-BCC4-6C7C65E2AB5A
bucket=rb_1466615370_1466529185_25_9B9D1F9-8EA5-4C73-BCC4-6C7C65E2AB5A Detected directory manually copied into its database, causing id conflicts [path1='C:\splunk_indexes\MyIndex\db\db_1466615370_1466529185_25_9B9D1F9-8EA5-4C73-BCC4-6C7C65E2AB5A' path2='C:\splunk_cold_indexes\MyIndex\db\rb_1466615370_1466529185_25_9B9D1F9-8EA5-4C73-BCC4-6C7C65E2AB5A'].

You would need to figure out the bucket conflicting issue by removing or moving it to somewhere Splunkd doesn't know.