I ran into an issue today after restarting my deployment server. After restarting, it would no longer load the "Forwarder Management" page, stating that there was an error in serverclass.conf. In the process of trying to find the error, I used btool to list the serverclass configurations. I noticed that there were a few stanzas located in "/opt/splunk/etc/apps/search/local/serverclass.conf". I never created this file and it only had about 6 stanzas in it, while my main serverclass.conf file in etc/system/local has over 80 stanzas.
I'm trying to prevent this issue from repeating, but I have no idea how the serverclass.conf file in etc/apps/search/local even came to exist. Any ideas?
If you use the Add Inputs wizard to create remote inputs, it will edit the etc/apps/search/local/serverclass.conf
I'm not sure why. I am also pretty sure that any changes made through the CLI will also be saved in etc/apps/search/local/serverclass.conf and not etc/system/local/serverclass.conf
I have two serverClasses and their associated stanzas defining the corresponding Deployment apps that appeared in ~/etc/apps/search/local/serverclass.conf and I did not do either of these two things. I have no idea why they landed there. That feels buggy to me. There must be something you can do before going to Forwarder management that causes it to place newly-defined serverClass(es) in the Search app's local.
The deployment server hasn't ended up as a deployment client, we also have it blacklisted. We did deploy a search app via deployment-apps to our search heads, but later deleted it. The search app was located under etc/deploymentapps/search-2. That stanza, or anything related to it doesn't show up in the suspicious serverclass.conf file.