Deployment Architecture

Splunk Server OS Security patching

maheshnc
Path Finder

Hello,

Our operations team is supposed to perform OS Security patching on indexer cluster, search head, Heavy Forwarders, deployment server and licence master, I want to know, as a Splunk admin what are the prechecks and post-checks need to be performed? for example, do we need to take backup etc.

thanks in advance.

0 Karma
1 Solution

maheshnc
Path Finder
0 Karma

kiran_panchavat
SplunkTrust
SplunkTrust

@maheshnc 

In addition to the other steps, make sure to back up the /opt/splunk/bin directory on the Heavy Forwarders. If any custom scripts were placed directly in /opt/splunk/bin, it’s essential to include this directory in your pre-patching backup.

Did this help? If yes, please consider giving kudos, marking it as the solution, or commenting for clarification — your feedback keeps the community going!
0 Karma

PrewinThomas
Motivator

@maheshnc 

General best practices are,

Backups & Config Safety first
-Back up $SPLUNK_HOME/etc (configs, apps, knowledge objects).
-Back up critical KV Store collections
-VM snapshots or system‑level backups in case rollback is needed.

For Indexer Cluster:
-Put the cluster into maintenance mode.
-Better patch one peer at a time.

Post Checks
Service Validation
-Confirm Splunk service is running
-Verify web UI(Applicable ones) and CLI access

Cluster Health
-Indexer Cluster - all peers should be Up and In‑Sync.
-SHC - all members should be Up and Ready
-Disable maintenance mode once all peers are patched


Data Flow
-Perform simple searches (index=YOUR_INDEX | head 5) to confirm indexing is happening.
-Check forwarders are still connected (Settings → Forwarder Management)

Regards,
Prewin
If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!

0 Karma

maheshnc
Path Finder

Thanks!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...