Splunk Enterprise

How to fix postgres sidecar collation errors

las
Contributor

Hi.

I recently upgraded my Deployment Server and thought I would look into the Edge Processor.

Unfortunately it is not very stable in my environment, and I see a lot of errors:

2026-06-10 16:45:57.523 CEST [53166] WARNING:  database "acies_config_service" has a collation version mismatch
2026-06-10 16:45:57.523 CEST [53166] DETAIL:  The database was created using collation version 2.28, but the operating system provides version 2.34.
2026-06-10 16:45:57.523 CEST [53166] HINT:  Rebuild all objects in this database that use the default collation and run ALTER DATABASE acies_config_service REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.

 error: error creating database -- Schema file for setting up database for sidecar service
--
-- Version: 1.3
-- Date: 02/09/2026
--
-- Change Log:
-- Version 1.3 (02/09/2026):
--   - Added database for ep_spl2_preview service.
-- Version 1.2 (02/26/2026):
--   - Added database for agent_manager service.
-- Version 1.1 (09/02/2025):
--   - Updated postgres server side per db connection limit for all databases.
-- Version 1.0 (05/17/2024):
--   - Initial schema creation for search_metadata, kvstore, acies_config_service, and opamp_service.

CREATE DATABASE search_metadata WITH OWNER = "do.root.splunk" CONNECTION LIMIT 10;
CREATE DATABASE kvstore WITH OWNER = "kvstore.pdl.splunk" TEMPLATE = template0 LC_COLLATE = 'C' LC_CTYPE = 'C' CONNECTION LIMIT 20;
CREATE DATABASE acies_config_service WITH OWNER = "acc.dbuser.splunk" CONNECTION LIMIT 10;
CREATE DATABASE opamp_service WITH OWNER = "oas.root.splunk" CONNECTION LIMIT 10;
CREATE DATABASE agent_manager WITH OWNER = "am.root.splunk" CONNECTION LIMIT 10;
CREATE DATABASE ep_spl2_preview WITH OWNER = "ep_preview.root.splunk" CONNECTION LIMIT 10;
: pq: template database "template1" has a collation version mismatch
	Failed to create database
	failed to create database
	Failed to bootstrap client artifact
   log_level: ERROR
   msg: Failed to bootstrap databases in standalone 

I have found a lot of suggestions that the databases shoul be rebuild - from splunk.my.site.com/customer/s/article/database-has-a-collation-version-mismatch for instance, where they give these instructions:

psql -h /tmp -p 6432 -d postgres -c "ALTER DATABASE postgres REFRESH COLLATION VERSION;"
psql -h /tmp -p 6432 -d postgres -c "REINDEX DATABASE postgres;"
psql -h /tmp -p 6432 -d phantom -c "ALTER DATABASE phantom REFRESH COLLATION VERSION;"
psql -h /tmp -p 6432 -d phantom -c "REINDEX DATABASE phantom;"

 

Now I'm unsure how to run these commands (I do not have root nor sudo for anythin outside /opt/splunk and even there limited) and exactly for which databases should these commands be run.

 

Hopefully someone out here has an answer, otherwise I'll create a case and post their answer.

Kind regards

las

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...

Monitoring AI Agents with Splunk Observability Cloud

Let’s say I’m running a travel planning AI app in production. A user asks for three concise hotel options in ...

[Puzzles] Solve, Learn, Repeat: Tiling

This puzzle (first published here) is based on finding groups of tessellated tiles (inspired by floor tiles I ...