Deployment Architecture

Change forwarder password in a large deployment

DrewO
Splunk Employee
Splunk Employee

Hello, If I wanted to deploy out a large number of forwarders, what is the best practice to change the default splunkd password?

Thanks!

Anyone?!?! Anyone?!?!

1 Solution

I_am_Jeff
Communicator

You've probably solved your issue, but...

I've deployed to a hundred or so Linux systems manually[1] via scripting. On my UFs there is only one account, admin, so I just copy the entire passwd file there.

Here's a few lines from my install script. I've previously copied files to /tmp. Running as the Splunk user (uid=1984), in $SPLUNK_HOME, any my semi-normal user, these if statements are run:

# Splunk section
if [ "$(id -u)" = "1984" ] ; then
tar zxvf /tmp/my-splunkuf.tgz
cp /tmp/passwd splunkforwarder/etc
chmod 600 splunkforwarder/etc/passwd
/opt/splunk/splunkforwarder/bin/splunk start --accept-license
exit 0
fi

(Another comment, # is messing up the formatting) sudo-priv-user section
if [ "$(id -u)" = "10101" ] ; then
sudo /opt/splunk/splunkforwarder/bin/splunk enable boot-start -user splunk
sudo chmod a+r /var/log/messages
sudo rm /tmp/passwd
rm /tmp/splunkme
rm /tmp/my-splunkuf.tgz
rm /tmp/adduser-splunk
fi

I have a custom tarball that I give to other groups for installation. Among other things, it includes a custom outputs.conf pointing at my indexers and $SPLUNK_HOME/splunkforwarder/bin is included in the Splunk user's $PATH in .profile. The other groups may have their own preferred password.

[1]"Manually" is misleading. I run a few shell scripts that copy files to multiple systems targetted as UFs, log in and run those scripts, then clean up after themselves. SSH keys are enabled so I don't have to enter my password hundreds of times.
If you understand this example, then you know what I mean.

for i in $( cat new-UF-list ) ; do
scp adduser-splunk splunkme file1 file2 file3 ${i}/tmp
ssh $i /tmp/adduser-splunk
ssh splunk@${i} /tmp/splunkme
ssh $i /tmp/splunkme
done

View solution in original post

I_am_Jeff
Communicator

You've probably solved your issue, but...

I've deployed to a hundred or so Linux systems manually[1] via scripting. On my UFs there is only one account, admin, so I just copy the entire passwd file there.

Here's a few lines from my install script. I've previously copied files to /tmp. Running as the Splunk user (uid=1984), in $SPLUNK_HOME, any my semi-normal user, these if statements are run:

# Splunk section
if [ "$(id -u)" = "1984" ] ; then
tar zxvf /tmp/my-splunkuf.tgz
cp /tmp/passwd splunkforwarder/etc
chmod 600 splunkforwarder/etc/passwd
/opt/splunk/splunkforwarder/bin/splunk start --accept-license
exit 0
fi

(Another comment, # is messing up the formatting) sudo-priv-user section
if [ "$(id -u)" = "10101" ] ; then
sudo /opt/splunk/splunkforwarder/bin/splunk enable boot-start -user splunk
sudo chmod a+r /var/log/messages
sudo rm /tmp/passwd
rm /tmp/splunkme
rm /tmp/my-splunkuf.tgz
rm /tmp/adduser-splunk
fi

I have a custom tarball that I give to other groups for installation. Among other things, it includes a custom outputs.conf pointing at my indexers and $SPLUNK_HOME/splunkforwarder/bin is included in the Splunk user's $PATH in .profile. The other groups may have their own preferred password.

[1]"Manually" is misleading. I run a few shell scripts that copy files to multiple systems targetted as UFs, log in and run those scripts, then clean up after themselves. SSH keys are enabled so I don't have to enter my password hundreds of times.
If you understand this example, then you know what I mean.

for i in $( cat new-UF-list ) ; do
scp adduser-splunk splunkme file1 file2 file3 ${i}/tmp
ssh $i /tmp/adduser-splunk
ssh splunk@${i} /tmp/splunkme
ssh $i /tmp/splunkme
done

kristian_kolb
Ultra Champion

If you are talking about setting an initial password at installation time, I guess that you could add

<path>/bin/splunk edit user admin -password <your new passswd> -auth admin:changeme

to your install/post-install script. However this depends on how you choose to deploy and on what platform. The bigger issue is how to change it at a later date. Unfortunately there is no safe way of doing it via the DeploymentServer. Neither can you upgrade the forwarder that way.

Sorry,

Kristian

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Index This | What travels the world but is also stuck in place?

April 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Discover New Use Cases: Unlock Greater Value from Your Existing Splunk Data

Realizing the full potential of your Splunk investment requires more than just understanding current usage; it ...

Continue Your Journey: Join Session 2 of the Data Management and Federation Bootcamp ...

As data volumes continue to grow and environments become more distributed, managing and optimizing data ...