I am a new Splunk user. I have finished all the Splunk Admin training. (Splunk Fundamentals 1 & 2, Splunk Administration, and Splunk Data Administration) I am still processing all the information.
However, I've been tasked with developing a migration plan to move our current environment from Splunk 6.6.0 to Splunk 7.2.1. A fairly large task for a new Splunk user. I'm excited about it, and not afraid to take on the task. So just need some help in getting pointed in the right direction. For security reasons, I can't give out to any details, but I'll include what I can.
Splunk 6.6.0 environment:
What I need to gather from what I know:
Once all data is gathered, our goal is to ensure that users aren't having to straddle both environments to access their content. So, this means migrating a group with their index, and migrating the searches, reports, and alerts for that index.
If anyone has a basic migration project plan, it would be helpful for the newbie.
Thanks for your help.
When you say "migrate" do you mean you will be installing the new version of Splunk on the existing servers or on new servers? If the latter, why? What will the new platform(s) be? If you will be using new servers, consider an all-Linux environment. You'll thank me later.
The new Splunk 7.2.1 is installed and up and running. It is an all Linux environment. Think of it as a completely separate environment. It is processing other forwarders now. We want to completely decommission the 6.0 environment all together.
Current New Environment:(Not really new, just a new enterprise environment and all Redhat Linux)
2-lightweight forwarders (For syslogs only)
1-Dev box for testing heavy forwarders
1-Splunk ES server
2-Deployment servers: One used to push updates to the non ES search heads. The other one is where all the apps live. The second one pushes updates to the cluster master, deployer, light forwarders, heavy forwarders, and universal forwarders. It is also has the common applications directory.
3-Search head servers as a cluster
1-Cluster master that manages the index servers
15-index servers as a cluster
What you describe is more of a merger than a migration. While it's possible, it's more complicated. The main thing to look out for is bucket collisions. See https://docs.splunk.com/Documentation/Splunk/7.2.3/Installation/MigrateaSplunkinstance
To answer your other questions:
1. The serverclass.conf file will contain all of your forwarders, but won't contain all of them. Unfortunately, there's no one place that has all instances. See https://docs.splunk.com/Documentation/Splunk/7.2.3/InheritedDeployment/Introduction for tips on finding your Splunk server types.
2. The best way to get this information is to look at the input.conf files on each forwarder. Shouldn't be a problem since you only have two.
3. The GUI can show you this information. Export, however, is probably limited to printing to a PDF. Another option is to use REST. Search for
| rest /serviceNS/-/-/saved/searches to get a list of all saved searches. Filter by adding
search commands like `| search NOT eai:acl.owner=admin'. This can be exported.
4. If it's in $SPLUNK_HOME/etc/users it's private.
5. A search analyzer would be a great feature. There are probably apps which can help with this, but I can't think of any ATM. If you want to write your own, REST is the key. You can use the search from #3 to get the bodies of all searches can search them for key attributes of inefficient searches, like "index=*", "join", "transaction", etc.
6. I'm not sure about this one, but REST probably is a big part of the answer.
7. See the Migrate A Splunk Instance link above.
Thanks for the response. This really helps, and gets me pointed in the right direction. I figured I would have to write some rest searches for most of it. This is going to be a complex task for sure. Lots of data to pull together. I did find information on the "Data Governance" app.
I'm not sure if that will help, but I'll get it installed, and give it a try.
I was told this week by management that we will not migrate any of the indexed data. We are only going to migrate the AD Groups, Roles, users, capabilities, and user's data. We will also move the associated indexes, if they do not exist in the new system. So for now, I just need to match up the AD Group with the role and capabilities for the users, and the indexes associated with them. Also need information on how to migrate the data to the new system.
Apparently we only keep a few weeks of indexed data. So we are working on moving the users with the index and servers and logs at the same time, and then point the users to the new environment. No need for bucket moves.