AIX upgrade fails with error - Dependent module /usr/local/splunkforwarder/lib/libpcre.a(libpcre.so.1) could not be loaded

[sat94541]

Splunk Universal forwarder upgrade from version 5.0.4 to any 6.x on 'AIX' failed.

-The upgrade failed exactly the same way on two different systems - one was AIX 6.1 and the other was AIX 5.3.

-All of the following upgrade attempts failed
Upgrade from Splunk Universal Forwarder 5.0.4 > Splunk Universal Forwarder 6.0.1
Upgrade from Splunk Universal Forwarder 5.0.4 > Splunk Universal Forwarder 6.0.2
Upgrade from Splunk Universal Forwarder 5.0.4 > Splunk Universal Forwarder 6.0.3
Impact : Customer has over 100 Splunk Universal Forwarder to be upgraded, so impact is widespread.

This worked:
-In the past, they have upgraded ‘Splunk Universal Forwarder’ from 5.0.2 to 5.0.4 without issue.
-Fresh install of Splunk Universal forwarder 6.x works fine
-Full Instance of Splunk upgraded fine from 5.0.4 to 6.0.x

Below are the error messages when starting Splunk post-upgrade.

********** BEGIN PREVIEW OF CONFIGURATION FILE MIGRATION **********

Could not load program /usr/local/splunkforwarder/bin/splunkd:
Dependent module /usr/local/splunkforwarder/lib/libpcre.a(libpcre.so.1) could not be loaded.
Member libpcre.so.1 is not found in archive

********** END PREVIEW OF CONFIGURATION FILE MIGRATION **********

dmr2o@xxxx[QA-ILMT][/usr/local/splunkforwarder]ls -l total 232
-r--r--r-- 1 root system 510 Dec 13 16:25 README-splunk.txt
drwxr-xr-x 3 root system 4096 Dec 13 16:36 bin
-r--r--r-- 1 root system 57 Dec 13 16:28 copyright.txt
drwxr-xr-x 12 root system 4096 Dec 13 16:36 etc
-rw-r--r-- 1 root system 0 Dec 13 16:28 ftr
drwxr-xr-x 2 root system 256 Dec 13 16:36 include
drwxr-xr-x 3 root system 4096 Dec 13 16:36 lib

………….

[sat94541]

Try following steps

---Delete the $SPLUNK_HOME/lib
--untar the 6.1 and upgrade.
--After taking the above steps you wil lneed to restart the host box , we believe it to be some cache maintained by AIX.

Hope it helps.