topic Re: DB Connect App FAILS to pull entire data to Splunk. in All Apps and Add-ons

DB Connect App FAILS to pull entire data to Splunk.

joydeep741 — Wed, 30 Sep 2020 00:00:21 GMT

I have to pull 30 million rows from database to splunk using DB Connect App.

I have kept FETCH_SIZE as 20,000.

But my data pull(indexing) stops after 20million rows or so.

I see this error on splunk_app_db_connect_server.log files

2019-04-10 23:18:32.350 -0700 [QuartzScheduler_Worker-16] ERROR org.easybatch.core.job.BatchJob - Unable to write records java.io.IOException: HTTP Error 503: Service Unavailable at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112) at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89) at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36) at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203) at org.easybatch.core.job.BatchJob.call(BatchJob.java:79) at org.easybatch.extensions.quartz.Job.execute(Job.java:59) at org.quartz.core.JobRunShell.run(JobRunShell.java:202) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.

How to resolve this ?

Re: DB Connect App FAILS to pull entire data to Splunk.

harsmarvania57 — Thu, 11 Apr 2019 14:16:52 GMT

That error suggests problem with HEC (Http Event Collector), please check whether HEC port is available or not. I'll suggest to Reduce Fetch Size and increase Max Rows.

Re: DB Connect App FAILS to pull entire data to Splunk.

joydeep741 — Thu, 11 Apr 2019 15:14:26 GMT

@harsmarvania57
I get 80% of records but it fails towards the end. So if HEC port wasn't available I wouldn't get any record, right ?

Re: DB Connect App FAILS to pull entire data to Splunk.

harsmarvania57 — Thu, 11 Apr 2019 15:29:44 GMT

Splunk DB connect 3 uses HEC to ingest data into Splunk, 503 error indicates that HEC is overloaded and due to that you missed those events.

Re: DB Connect App FAILS to pull entire data to Splunk.

joydeep741 — Thu, 11 Apr 2019 15:38:23 GMT

@harsmarvania57
So what do you suggest to handle this ??
Do we need to upgrade the infra ? or do we need to change the values of max rows and Fetch size ?

Re: DB Connect App FAILS to pull entire data to Splunk.

harsmarvania57 — Thu, 11 Apr 2019 15:52:35 GMT

I'll suggest with less number of fetch size and increase Max rows (but not too much) and see how it performs.

Re: DB Connect App FAILS to pull entire data to Splunk.

joydeep741 — Fri, 12 Apr 2019 07:27:09 GMT

Tried with different combinations of Fetch Size and Max rows. But doesn't work.

Re: DB Connect App FAILS to pull entire data to Splunk.

harsmarvania57 — Fri, 12 Apr 2019 08:49:37 GMT

What is frequency of polling data from Database ? Is it Batch Mode or Rising Column mode ? What are server specification on which DB Connect is running ?