Getting Data In

Why am I unable to delete indexes from the Splunk Web? Why do I have to restart Splunk when I create a new Index from Splunk Web?

dshakespeare_sp
Splunk Employee
Splunk Employee

Customer reported several issue with Index Management using the Splunk Web:
- Unable to create new Indexes from Settings > Indexes > New Index (Splunk Web reports that a restart is required)
- Unable to create new Indexes from Data Inputs > Files & Directories > New (Splunk Web reports that a restart is required)
- Unable to delete Indexes from Settings > Indexes > Delete (Splunk Web will not accept any input apart from "Cancel")

1 Solution

dshakespeare_sp
Splunk Employee
Splunk Employee

On investigation TWO issues were found.

  • The "need a restart" type messages seen in the GUI suggested that the indexes.conf on disk had been manually changed while Splunk was running and Splunk had not been restarted so that the running version in memory matched the disk version.

This issue was supported by the following error in splunkd.log:

"unable to reload indexer after adding: reason='reload is not safe since a path has been deleted or modified for an index, or an index has been disabled. You must restart the Splunk Server, for your changes to take effect.'. Restart required." messages seen in splunkd.log.
  • splunkd.log also reported the error

    ERROR IndexConfig - stanza=osstats Required parameter=homePath not configured

Using "splunk btool indexes list --debug to locate the osstats stanza it was observed that in $SPLUNK_HOME/etc/system/local/inputs.conf, the following was declared

[osstats]
disabled = 1

Even though the stanza is disabled Splunk needs a homePATH coldPATH and thawedPATH to tell Splunk where the data files are. Without these stanzas Splunk sees the indexes.conf as invalid and may have problem configuring changes.

The stanza was changed to read

[osstats]
coldPath = $SPLUNK_DB/osstats/colddb
homePath = $SPLUNK_DB/osstats/db
thawedPath = $SPLUNK_DB/osstats/thaweddb
disabled = 1 

On restarting Splunk, the delete index functionality was returned to normal operation

View solution in original post

dshakespeare_sp
Splunk Employee
Splunk Employee

On investigation TWO issues were found.

  • The "need a restart" type messages seen in the GUI suggested that the indexes.conf on disk had been manually changed while Splunk was running and Splunk had not been restarted so that the running version in memory matched the disk version.

This issue was supported by the following error in splunkd.log:

"unable to reload indexer after adding: reason='reload is not safe since a path has been deleted or modified for an index, or an index has been disabled. You must restart the Splunk Server, for your changes to take effect.'. Restart required." messages seen in splunkd.log.
  • splunkd.log also reported the error

    ERROR IndexConfig - stanza=osstats Required parameter=homePath not configured

Using "splunk btool indexes list --debug to locate the osstats stanza it was observed that in $SPLUNK_HOME/etc/system/local/inputs.conf, the following was declared

[osstats]
disabled = 1

Even though the stanza is disabled Splunk needs a homePATH coldPATH and thawedPATH to tell Splunk where the data files are. Without these stanzas Splunk sees the indexes.conf as invalid and may have problem configuring changes.

The stanza was changed to read

[osstats]
coldPath = $SPLUNK_DB/osstats/colddb
homePath = $SPLUNK_DB/osstats/db
thawedPath = $SPLUNK_DB/osstats/thaweddb
disabled = 1 

On restarting Splunk, the delete index functionality was returned to normal operation

Get Updates on the Splunk Community!

New Cloud Intrusion Detection System Add-on for Splunk

In July 2022 Splunk released the Cloud IDS add-on which expanded Splunk capabilities in security and data ...

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...