Indexes.conf config stuck

[jessieb_83]

In Indexes.conf from the CM, I tried to set thawedHomePath to a volume, which I have since learned does not work.

I set the path from volume:cold back to $SPLUNK_DB, but no matter what I do the indexer will not acknowledge that I changed it back. It still thinks it's set to the volume. I modified it, commented it  out, deleted the whole indexes.conf file and loaded a manual one in the `/etc/system/local/indxes.conf  and nothing will un-stick it. Every time I start the indexer, the logs show it won't start because thawedHomePath is mapped to a volume still.

When I run ~\splunk btool indexes list --debug  it shows the thawedHomePath in question is configured correctly.

Has anyone ever experienced this before? Any suggestions on how to get it to accept the change? 

Running Splunk 9.2 on RHEL 8 with 1 CM and 2 IDXs clustered together. Fairly new deployment, still working the bugs out.

[gcusello]

Hi @jessieb_83 ,

there's no reason to insert thawedHomePath in a volume because it's a mount point to use when you have to remount discarded buckets:

hawedPath = <string>
* An absolute path that contains the thawed (resurrected) databases for the
  index.
* CANNOT contain a volume reference.
* Path must be writable.
* Required. Splunkd does not start if an index lacks a valid thawedPath.
* You must restart splunkd after changing this setting for the changes to
  take effect. Reloading the index configuration does not suffice.
* Avoid the use of environment variables in index paths, aside from the
  exception of SPLUNK_DB. See 'homePath' for additional information as
  to why.

as you can read at https://github.com/jewnix/splunk-spec-files/blob/master/indexes.conf.spec 

This is a manual action thet you do only on request, it isn't a continue action.

If you want to maintain in-line discarded logs, enlarge your retention period and maintain them in Cold state.

Ciao.

Giuseppe

[jessieb_83 · Accepted solution]

You are correct, @gcusello , I wouldn't normally. The only reason I was listing it was because the error I was getting at start up about it being incorrect, so I was trying to override it.

As it turns out, the error came from when I fixed it on the distributed indexes.conf file, it didn't actually SAVE the correction, so that peer-app kept overriding everything else I did with the wrong configuration. and it took me several hours of staring at the error before I actually saw it. 

So it was a carbon based error.