After upgrading to 6.5.0 on Windows, I receive error 1607 when restarting splunkd - how to fix?

[peterchow]

Dear all,

I tried to upgrade Splunk from 6.1.1 to 6.5 but I'm having some issues.

The first time, there is an error during installation and it caused splunkd to disappeared in service manager. After that, I tried to install again and it was successful. However there is another problem.

When I try to restart splunkd service, it show error 1067. I checked the log and how following message.

D:\Program Files\Splunk\var\lib\splunk\persistentstorage\seqno_db : unable to open database file
GetLastError(): 5

I tried to remove it and solve the issue but this message appears again when I restart the service.

May i know what problem is?

Thanks

[dineshraj9]

Try running this in command prompt in Splunk installation directory and then restart Splunk - $SPLUNK_HOME/bin/splunk clean locks

In your case -

D:\Program Files\Splunk\bin\splunk clean locks
D:\Program Files\Splunk\bin\splunk restart

[afret2007]

A 1067 error more often than not means you will need to go in to Windows registry and delete all key values referencing Splunk.. I had similar problem upgrading an older Splunk Version to a newer one. If you get a 1067 error and removal and installation does not fix issue, then only recourse I found that worked was going into the Window Registry (do not attempt if you are unfamiliar with manipulating Window Registry!) and do what I stated above. Make sure you uninstall Splunk first so the number of keys and values to delete are lower in number. Once done reboot machine and then do a new fresh install of Splunk v6.5. You should be good to go then. (No guarantees!)

[skoelpin]

I downvoted your answer because suggesting OP should "make changes to registry, uninstall splunk, reinstall, NO GUARANTEES" is extremely bad advise. OP needs to contact customer support rather than listening to a user with near zero reputation and experience.

[afret2007]

Fair enough but as stated it was as a comment and not an answer. I only mentioned that was the only way I got mine to work. I should of been more clear that it was how I got mine to finally work and not present it like it was the first and only way to fix the issue. You are right to call me on that. The official answer from Splunk Support is to run the clean locks command. I ran the official solution a couple of years ago and it did not work (lock file went away but 1067 error did not). I did state that if OP was not comfortable dealing with Windows Registry (I am and there are many cases when hardening a system it is required) to not attempt. The issue I had was an upgrade to early version of 6.x and not latest. Outside of the one time 1067 showed up Our Splunk system on Windows has run flawlessly for the 3000+ systems we monitor. I am not saying that Windows is better than Linux....I always preferred Linux over windows. It is just some organizations dictate the OS (especially government) and the Splunk Administrators have no choice but to use Windows and cannot make the conversion to the more stable Linux OS. You are correct Windows does have its drawbacks but I have found out in my experience with Splunk Windows (six years now) that official Splunk solutions given may work for Splunk Linux but many times not for Windows.

[skoelpin]

I'm assuming your on Windows since you have a D drive.. Avoid using WIndows, the upgrade process is so painful. I would recommend you open a support case

[afret2007]

Telling someone to "Avoid using Windows" is not an answer. If you are going to make such statements, keep it in the comment section of the individual's question. There are a lot of organizations where it is dictated to the Splunk administrators what OS is to be used and is beyond their control.

[skoelpin]

It appears that you did not read the post in its entirety as my answer stated to contact customer support to help with the upgrade process.. Making changes to the registry (As you suggested above) then having to restart the server is NOT a good idea at all, especially if the user is not 100% comfortable making those changes. Also, if the Splunk administrator has any legitimate Splunk experience would know running their indexer on a Windows machine is a very bad idea for reasons OP discussed in his question. So I suggest you gain more experience before chiming in for future questions