Splunk says it has "received event for unconfigured/disabled/deleted index=", except it does exist and ends up working. What gives?

[thisissplunk]

Created an index on the gui just fine. Configed up the forwarder's inputs.conf and props.conf. Moved data into the monitored directories on the forwarder. Splunkd on the forwarder says Splunk is reading the file fine.

However, the first event or so in the first monitored file (I assume) is triggering this in the search head gui:

"received event for
unconfigured/disabled/deleted index="

Again though, the index IS setup, and the "Current Size" is filling up in the Indexes gui page. If I wait long enough, events will start populating the index from files that came at LEAST after the first file. Currently waiting to confirm if the first file actually is getting indexed at all (it's huge).

What is going on here? Why does Splunk seem to think the index doesn't exist when it does, and when it's working?

[mayurr98 · Accepted solution]

Hi
This message comes when you do not define indexes.conf

1) first login to your indexer using CLI
Go to /opt/splunk/etc/apps/seach/local/
And look for indexes.conf
If you see your index in indexes.conf then note it down or else create the new index again.

2) login to forwarder and go to /opt/splunkforwarder/etc/system/local/inputs.conf

And in the monitor stanza and look for
index = your_index

Name of “your_index” must be exactly same as the one you have created on indexer.

I hope this helps.
Let me know if it works!

[thisissplunk]

I'll give this to you because it was ultimately right. We never set up the index on the 2nd peer. We thought that the forwarder would be smart enough to NOT send data to a peer that didn't have the index. Thanks.

[somesoni2]

You've standalone Splunk instance (acting as SH and Indexer both) and that's where you created the index and forwarding you data to?

[thisissplunk]

Yes, correct.

[esix_splunk]

Where are you checking that the index exists? On SH? Or on the Indexer?

[thisissplunk]

By checking, I'm looking at it in the gui of the SH/Indexer box under "Settings->Indexes"

[esix_splunk]

What index does it say is missing? Or is the above the actual message? If its the actual message you need to check your inputs, I think you have a invalid inputs defined for an index...

[thisissplunk]

The message says the indexer that is missing is the same one that exists under "Settings->Indexes". The index setting in the inputs.conf is correct and matches.

To reiterate, events do end up populating the index, but the error shows up during the first time I move a file into the monitoring directory. It happens each time I make a new index and monitor new files.

[saurabh_tek11]

@thisissplunk seems like system resources on your single instance splunk box is less.
Are you seeing system sluggishness as well ?