DB Connect 3.1.4 - Unable to start server

[skirven]

Hi! I'm trying to get a DB Connect 3.1.4 running, and what's strange is I had one working, but it stopped. Now, even with a fresh installation, I can't get it to start. I also tried from another server, and it's also failing It looks from the logs like the Task Server is failing with a 404. I do have Java in the path, so I can't see why this wouldn't be working. Any help would be appreciated.

020-02-03 16:16:55.187 -0500 45786@ [main] INFO  org.eclipse.jetty.server.AbstractConnector - Started application
@62b26bef{HTTP/1.1,[http/1.1]}{127.0.0.1:9998}
2020-02-03 16:16:55.187 -0500 45786@[main] ERROR io.dropwizard.cli.ServerCommand - Unable to start server, shutti
ng down
javax.servlet.ServletException: io.dropwizard.jersey.setup.JerseyServletContainer-71daf5d6@74ee0fd6==io.dropwizard.jersey.setup.
JerseyServletContainer,jsp=null,order=1,inst=false
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:691)
        at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:427)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:760)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:374)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:848)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at com.codahale.metrics.jetty9.InstrumentedHandler.doStart(InstrumentedHandler.java:103)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:403)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.server.handler.StatisticsHandler.doStart(StatisticsHandler.java:252)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.server.Server.start(Server.java:419)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.server.Server.doStart(Server.java:386)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:53)
        at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:44)
        at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
        at io.dropwizard.cli.Cli.run(Cli.java:75)
        at io.dropwizard.Application.run(Application.java:93)
        at com.splunk.dbx.server.bootstrap.TaskServerStart.startTaskServer(TaskServerStart.java:121)
        at com.splunk.dbx.server.bootstrap.TaskServerStart.streamEvents(TaskServerStart.java:72)
        at com.splunk.modularinput.Script.run(Script.java:66)
        at com.splunk.modularinput.Script.run(Script.java:44)
        at com.splunk.dbx.server.bootstrap.TaskServerStart.main(TaskServerStart.java:150)
Caused by: java.lang.RuntimeException: com.splunk.HttpException: HTTP 404 -- Could not find object id=http
        at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollectorFactory.create(HttpEventCollectorFactory.java:71)
        at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollectorFactory.create(HttpEventCollectorFactory.java:52)
        at com.splunk.dbx.server.TaskServer.createHecService(TaskServer.java:487)
        at com.splunk.dbx.server.TaskServer.access$300(TaskServer.java:158)
        at com.splunk.dbx.server.TaskServer$2.configure(TaskServer.java:238)
        at org.glassfish.hk2.utilities.binding.AbstractBinder.bind(AbstractBinder.java:187)
        at org.glassfish.jersey.model.internal.CommonConfig.configureBinders(CommonConfig.java:676)
        at org.glassfish.jersey.model.internal.CommonConfig.configureMetaProviders(CommonConfig.java:641)
        at org.glassfish.jersey.server.ResourceConfig.configureMetaProviders(ResourceConfig.java:829)
        at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:453)
        at org.glassfish.jersey.server.ApplicationHandler.access$500(ApplicationHandler.java:184)
        at org.glassfish.jersey.server.ApplicationHandler$3.call(ApplicationHandler.java:350)
        at org.glassfish.jersey.server.ApplicationHandler$3.call(ApplicationHandler.java:347)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
        at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:255)
        at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:347)
        at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:392)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:177)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:369)
        at javax.servlet.GenericServlet.init(GenericServlet.java:244)
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:670)
        ... 45 common frames omitted
Caused by: com.splunk.HttpException: HTTP 404 -- Could not find object id=http
        at com.splunk.HttpException.create(HttpException.java:84)
        at com.splunk.DBXService.sendImpl(DBXService.java:131)
        at com.splunk.DBXService.send(DBXService.java:43)
        at com.splunk.HttpService.get(HttpService.java:154)
        at com.splunk.Entity.refresh(Entity.java:381)
        at com.splunk.Entity.refresh(Entity.java:24)
        at com.splunk.Resource.validate(Resource.java:186)
        at com.splunk.Entity.validate(Entity.java:462)
        at com.splunk.Entity.getContent(Entity.java:157)
        at com.splunk.Entity.size(Entity.java:416)
        at java.util.HashMap.putMapEntries(HashMap.java:501)
        at java.util.HashMap.<init>(HashMap.java:490)
        at com.splunk.dbx.model.repository.JsonMapperEntityResolver.apply(JsonMapperEntityResolver.java:34)
        at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollectorFactory.enableGlobalHec(HttpEventCollectorFactory.jav

[skirven · Accepted solution]

After much research, I figured this one out. I had installed the HEC app on the same server. It binds to port 8088, which is not going to allow the DB Connect app to run. I disabled HEC, and all was well.
Thanks!
Stephen

[jokertothequinn]

Hi, can you help me know what were you running on port 8088 except the HEC?

[isoutamo]

On Linux you could use command “ss -napt” to see what processes are running an in which port.

[deepakdora]

Hi Skriven! 

What did you exactly mean by disabling the HEC app. Can you give some more details about the app you are talking about?

[isoutamo]

Hi

as the newer DB Connections use internally HEC it try to bind the same port which was used by HEC. For that reason it didn’t start. There are two ways to solve it, just switch port of HEC or DBX internally. Or as @skirven did, disable HEC on this node and enable it somewhere else.

r. Ismo