All Apps and Add-ons

DBConnect migration from 2.4 to 3.1 troubleshooting

aurelien_cattez
New Member

Hello,

I am working on a Splunk Entreprise infrastructure and I had to migrate DBConnect from 2.4 to 3.1 on a single instance recently, and I had a lot of trouble to make it work the intended way.

I spent many hours trying to figure out what was wrong and as I saw no documentation for troubleshooting the migration, here is something to help you out.

Installing the app :

The documentation states that you should install the new app then launch the new migration script. In my case it did not work to unzip the new app on top of the old one on the search head, I had a lot of missing changes : 8 changes proposed by the script against 600 with the intended way. The app also didn’t behaved the way it should have. Moving the app directory to a different name then unzipping the new app then launching the migration script also did not work.

I had success with simply updating from the web interface and then launching the script.

Launching the script

The script should be ran with the port where the management port of the component where DBConnect is installed is. Script runs by default on port 8089, so in my case I was trying to run it on the Deployment server management port. I received these errors :

File "./app_migration.py", line 930, in

if "shc_deployer" in client.Entity(service, "server/roles").content.role_list:
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 872, in init
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 1011, in refresh
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 981, in get
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 738, in get
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 286, in wrapper
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 68, in new_f
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 660, in get
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 1150, in get
File "/opt/splunk/search_head/etc/apps/splunk_app_db_connect/bin/splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 1205, in request
splunklib.binding.HTTPError: HTTP 404 Not Found -- Application does not exist: splunk_app_db_connect

Changing the port in the command with -port XXXX option worked for me.

HTTP Event Collector

Apparently DBConnect 3 uses the HTTP Event Collector to write into the indexes, at least in my case. It uses port 8088. If that port is occupied, it might seems like everything works, but nothing is written to the indexes. You might encounter these errors in the dbx logs :

2018-06-26 12:01:58.966 +0200 [QuartzScheduler_Worker-8] ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch
java.io.IOException: HTTP Error 401: Unauthorized
at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112)
at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89)
at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36)
at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203)
at org.easybatch.core.job.BatchJob.call(BatchJob.java:79)
at org.easybatch.extensions.quartz.Job.execute(Job.java:59)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)


2018-06-26 12:44:16,442 ERROR [5b3219006c7f6ef435a910] config:138 - [HTTP 401] Client is not authenticated
Traceback (most recent call last):
File "/opt/splunk/search_head/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/config.py", line 136, in getServerZoneInfo
return times.getServerZoneinfo()
File "/opt/splunk/search_head/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/times.py", line 158, in getServerZoneinfo
serverStatus, serverResp = splunk.rest.simpleRequest('/search/timeparser/tz')
File "/opt/splunk/search_head/lib/python2.7/site-packages/splunk/rest/init.py", line 530, in simpleRequest
raise splunk.AuthenticationFailed
AuthenticationFailed: [HTTP 401] Client is not authenticated

Port was occupied in my case so I changed the HTTP Event Port, and restarted Splunk, it worked after that.

I hope that will prove useful for some of you 🙂 !

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...