Current search head is on a VM. I have set up a new search head now which is on a physical server. Both have search peers set up correctly. The current VM search head has all of the user-specfiic settings, dashboards, searches, views, etc configured. The new physical search head does not.
What specific files do I need to move from the first search head (VM) to the second search head (physical)? (that is, which files under $SPLUNKHOME/etc need to be moved, and are there any files NOT under $SPLUNKHOME/etc which need to be moved?
Also, the first Search head is also the license server. What is the best way to move the license over from the first search head to the second and then remove from the first? Do make the second search head the license master, install license there, then re-point my indexers to the new server?
Each Search Head has its own private copy of $SPLUNK_HOME/etc/system.
Search Head Pooling allows for synchronized sharing of $SPLUNK_HOME/etc/users and $SPLUNK_HOME/etc/apps via shared storage.
Authentication(local, LDAP etc..) must be setup on each Search Head individually.
$SPLUNK_HOME/etc/passwd (if using local authentication)
Alternatively to setting up pooling as detailed above , you could "rsync" between your 2 Search heads to keep $SPLUNK_HOME/etc/users and $SPLUNK_HOME/etc/apps synchronized and the auth related config files in sync.
Regarding the License Server refactoring , I haven't done a migration as you describe, but I don't see any caveats with your approach.
I'll just add that I prefer to use a DNS CName for my Splunk License Server so that I don't need to update my license client's "master_uri" value if I were to move the license server to a new host, I can just update the DNS CName record.