Getting Data In

How to run multiple universal forwarders on a single Linux host?

tusharsaran1
Path Finder

I am trying to install 2 universal forwarders on a single Linux host.
I read a few articles and changed the httpport and mgmtHostPort in the web.conf file in $SPLUNK_HOME/etc/system/local.
I also changed serverName in server.conf file.
On trying to start the new splunk instance, it gives below error:

The splunk daemon (splunkd) is already running.            [FAILED]

Is there something else that needs to be changed or is there a different method of running 2 universal forwarders on 1 host?

1 Solution

mattymo
Splunk Employee
Splunk Employee

DISCLAIMER: AS FAR AS I KNOW YOU WILL NOT MAKE FRIENDS IN SPLUNK SUPPORT IF YOU DO THIS IN PROD AND HAVE ISSUES. NOT GOING AS FAR AS SAYING IT IS UNSUPPORTED (STILL LOOKING TO CONFIRM), BUT YOUR MILEAGE MAY VARY IF YOU HAVE ISSUES AND LOOK TO SUPPORT TO BAIL YOU OUT. USE PARALLELIZATION IF YOU NEED MORE PIPELINES/THRUPUT. CREATE APPS TO SEGREGATE CONFIG MANAGEMENT

http://docs.splunk.com/Documentation/Splunk/6.5.3/Capacity/Parallelization
http://dev.splunk.com/view/get-started/SP-CAAAESC

I was able to successfully install 2 6.5.3 UFs on a Centos 7 box by doing the following:

Step 1 - As root, cd to /opt and wget UF download.

Step 2 - tar -zxvf splunkforwarder-6.5.3-36937ad027d4-Linux-x86_64.tgz

Step 3 - mv splunkforwarder splunkforwarder2

Step 4 - tar -zxvf splunkforwarder-6.5.3-36937ad027d4-Linux-x86_64.tgz

Step 5 - chown -R splunker:splunker *

Step 6 - cd /opt/splunkforwarder2/etc/system/local and vi server.conf

[general]
serverName = splunkforwarder2

Step 7 - vi web.conf

[settings]
httpport = 9000
mgmtHostPort = 127.0.0.1:8099

Step 8 - cd /opt/splunkforwarder/bin and run ./splunk start --accept-license --answer-yes

Step 9 - cd /opt/splunkforwarder2/bin and run ./splunk start --accept-license --answer-yes

Step 10 - verify with ps -ef | grep splunkd

[splunker@n00b-splkufw-01 local]$ ps -ef | grep splunkd
splunker 9992 1 0 18:16 ? 00:00:00 splunkd -p 8089 start
splunker 10000 9992 0 18:16 ? 00:00:00 [splunkd pid=9992] splunkd -p 8089 start [process-runner]
splunker 10073 1 0 18:17 ? 00:00:00 splunkd -p 8099 start
splunker 10081 10073 0 18:17 ? 00:00:00 [splunkd pid=10073] splunkd -p 8099 start [process-runner]
splunker 10145 9431 0 18:21 pts/0 00:00:00 grep --color=auto splunkd

One slight differences in your configs, I left out the [default] stanza in server.conf, not needed.

Couple things to note, I started the UFs in sequence. and I have confirmed I can run ./btool fine:

cd /opt/splunkforwarder/bin

[splunker@n00b-splkufw-01 bin]$ ./splunk btool server list general --debug
/opt/splunkforwarder/etc/system/local/server.conf [general]
/opt/splunkforwarder/etc/system/default/server.conf access_logging_for_phonehome = true
/opt/splunkforwarder/etc/system/default/server.conf allowRemoteLogin = requireSetPassword
/opt/splunkforwarder/etc/system/default/server.conf connectUsingIpVersion = auto
/opt/splunkforwarder/etc/system/default/server.conf defaultHTTPServerCompressionLevel = 6
/opt/splunkforwarder/etc/system/default/server.conf hangup_after_phonehome = false
/opt/splunkforwarder/etc/system/default/server.conf instanceType = download
/opt/splunkforwarder/etc/system/default/server.conf listenOnIPv6 = no
/opt/splunkforwarder/etc/system/default/server.conf parallelIngestionPipelines = 1
/opt/splunkforwarder/etc/system/local/server.conf pass4SymmKey =
/opt/splunkforwarder/etc/system/local/server.conf serverName = n00b-splkufw-01.n00blab.local
/opt/splunkforwarder/etc/system/default/server.conf sessionTimeout = 1h
/opt/splunkforwarder/etc/system/default/server.conf skipHTTPCompressionAcl = 127.0.0.1 ::1
/opt/splunkforwarder/etc/system/default/server.conf tar_format = gnutar
/opt/splunkforwarder/etc/system/default/server.conf useHTTPClientCompression = false
/opt/splunkforwarder/etc/system/default/server.conf useHTTPServerCompression = true

cd /opt/splunkforwarder2/bin

[splunker@n00b-splkufw-01 bin]$ ./splunk btool server list general --debug
/opt/splunkforwarder2/etc/system/local/server.conf [general]
/opt/splunkforwarder2/etc/system/default/server.conf access_logging_for_phonehome = true
/opt/splunkforwarder2/etc/system/default/server.conf allowRemoteLogin = requireSetPassword
/opt/splunkforwarder2/etc/system/default/server.conf connectUsingIpVersion = auto
/opt/splunkforwarder2/etc/system/default/server.conf defaultHTTPServerCompressionLevel = 6
/opt/splunkforwarder2/etc/system/default/server.conf hangup_after_phonehome = false
/opt/splunkforwarder2/etc/system/default/server.conf instanceType = download
/opt/splunkforwarder2/etc/system/default/server.conf listenOnIPv6 = no
/opt/splunkforwarder2/etc/system/default/server.conf parallelIngestionPipelines = 1
/opt/splunkforwarder2/etc/system/local/server.conf pass4SymmKey =
/opt/splunkforwarder2/etc/system/local/server.conf serverName = splunkforwarder2
/opt/splunkforwarder2/etc/system/default/server.conf sessionTimeout = 1h
/opt/splunkforwarder2/etc/system/default/server.conf skipHTTPCompressionAcl = 127.0.0.1 ::1
/opt/splunkforwarder2/etc/system/default/server.conf tar_format = gnutar
/opt/splunkforwarder2/etc/system/default/server.conf useHTTPClientCompression = false
/opt/splunkforwarder2/etc/system/default/server.conf useHTTPServerCompression = true

Hope that helps! Learning purposes only!!!

- MattyMo

View solution in original post

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...