Getting configuration for forwarder

[trflesher]

“I am working with a customer who is a licensed and valid support contract holder with Splunk. They are currently running a forwarder within their XenServer environment on Linux (CentOS). We were trying to migrate the machine to VMware ESXi but have experienced issues with the conversion process.

We plan to build a new Windows based server instead for the forwarders; however, are unsure how to get configuration data out of their current forwarder to the new one.”

[rajneeshsaraswa]

Hi,
Is it possible if you any details how to configure the splunk in Xenserver.
I am almost novice to Xenserver and Splunk.

[jconger]

You can stay with CentOS if you like, or you can use Windows. Either way, the first step is to install the Splunk Universal Forwarder on the machine http://www.splunk.com/download/universalforwarder

If you still have access to the old CentOS VM, you can copy the configuration from it to your new machine.

By default, the Splunk Universal Forwarder will install to /opt/splunkforwarder on CentOS. All the XenServer configuration files will be located at /opt/splunkforwarder/etc/apps/TA-XS60-Server. Just copy this entire folder to your new machine.

If you are going to use Windows instead of CentOS, you will need to modify the inputs.conf file located in this folder. You need to change all the forward slashes "/" to backslashes "\"

[lguinn2]

Just a note: since Splunk runs a service on Windows boxes, there is no need for something like "boot start"

[lguinn2]

And reinstall Splunk, of course - the Windows binaries are different! But if you re-install Splunk over an existing installation, it should preserve all the customer configurations. [Unless they broke the rules and edited files in the default folders.... 😞 ]

[jconger]

Once you copy the files and folders onto your Windows box under %ProgramFiles%\Splunk\etc\apps, you may need to change the forward slashes "/" in the various inputs.conf files to back slashes "\". That is really all you need to do.

[trflesher]

So the customer would like to eliminate Linux now and just use Windows. They already have Windows boxes running splunk for other things. I have been able to copy all the files from the splunk home folder on the CentOS vm. How would I get one of the existing windows servers to recognize that config and perform the functions that were on the CentOS box? Is this possible? Thanks!!

[jconger]

If you do not want to monitor XenServer, then no, you do not need to worry about the XenServer configuration files. If you do still want to monitor XenServer pools, then you can copy the XenServer configuration files anywhere you have a universal forwarder that can reach the XenServer pool master via TCP/IP - it doesn't matter where the XenServer collector actually "lives".

[trflesher]

A huge thank you to both lguinn and Jason (jconger) on this!! I appreciate it very much.

So I am assuming since the new CentOS vm will be running in VMware instead, I don't need to worry about the XenServer configuration files? Just double checking before I hopefully wrap up getting this going.

[lguinn2]

It is easy enough to grab the configuration data, but you have to know how the forwarder is configured 🙂

All Splunk configuration files are simple text. If Splunk is installed in the $SPLUNK_HOME directory, all configuration files can be found under $SPLUNK_HOME/etc However, this includes default configurations as well, which you should NOT copy from installation to installation - especially across OSes. When you look at the directory structure beneath $SPLUNK_HOME/etc, you will see a lot of folders named default that contain those default configurations. Under $SPLUNK_HOME/etc/apps you may also see entire apps (sets of configuration files) that will not be appropriate for the new environment.

OTOH, if the customer is using the Splunk Deployment Server to configure their forwarder(s), you just need to change the settings on the Deployment Server.

Also, I feel compelled to ask, why change the underlying OS? It complicates the conversion in more ways than one, and it costs more for the OS license (Splunk doesn't care). In the most common case, the Splunk forwarder runs on a production machine to collect and forward local data. Obviously, OS is determined by the production machine (web server, db server, etc etc). In this case, it sounds like the Splunk forwarder is the main service provided by the server - and you should be able to use any supported OS.

Finally, it actually doesn't matter if the customer has a support contract or not. The configuration files, etc, all work the same (especially for forwarders) regardless of the license. But it does mean that you can contact Splunk Support with specific questions about this migration!

[lguinn2]

Just move over the $SPLUNK_HOME; no install needed

the only thing you probably should do after that, is this command

sudo $SPLUNK_HOME/bin/splunk boot enable boot-start -user splunkX

where splunkX is the Linux account name that is running Splunk. This sets Splunk to start at boot. More info here

http://docs.splunk.com/Documentation/Splunk/6.0.1/admin/ConfigureSplunktostartatboottime

[trflesher]

Hmmm .. definitely open to try that. Is there any type of "install" we have to run ahead of time on the new CentOS VM built from scratch specific to Splunk? Or do I just need to move over that $SPLUNK_HOME ?

[lguinn2]

Sounds like a good enough reason to me. 🙂

Although if you build a new CentOS VM from scratch, you could simply copy $SPLUNK_HOME to the new machine.

Splunk does not spray stuff around the disk. tar up $SPLUNK_HOME on the old box and move it to the other - as long as they are both CentOS, you are fine.

[trflesher]

Thanks!! As far as OS goes, the challenge was converting the machine "as is" from a Xenserver vm to vmware vm. This was what we planned to do. Since it didn't work and all efforts to get around the problem of converting the CentOS were exhausted, we headed down this path. The customer has very limited knowledge on the CentOS itself and the implementation, so it's about supportability more than anything.