All Apps and Add-ons

Splunk DB Connect App: How to troubleshoot why db input fails to fetch results sometimes?

Motivator

Hi, I am trying to troubleshoot why db input fails to fetch results sometimes.

We have a db connect input which runs every hour and it fails occasionally. When looking at splunk internal logs, I have seen two kinds of messages in in splunkappdbconnectjobmetrics.log file. What is the meaning of readcount and write_count fields

SUCCESS MESSAGE:

2020-01-25 07:39:00.683 -0500 INFO  c.s.dbx.server.task.listeners.JobMetricsListener - action=collect_job_metrics connection=Vertica_Prod_LongRun_Conn jdbc jdbc_url=null db_read_time=0 hec_upload_time=3 hec_record_process_time=0 format_hec_success_count=1 hec_upload_bytes=647 status=COMPLETED input_name=DWH.Vertica.UnmatchedImpressionsMonitoring batch_size=1000 error_threshold=N/A is_jmx_monitoring=false start_time=2020-01-25_07:16:00 end_time=2020-01-25_07:39:00 duration=1380682 read_count=19 write_count=19 filtered_count=0 error_count=0

FAILED MESSAGE:

2020-01-25 06:27:45.809 -0500 INFO  c.s.dbx.server.task.listeners.JobMetricsListener - action=collect_job_metrics connection=Vertica_Prod_LongRun_Conn jdbc_url=null status=FAILED input_name=DWH.Vertica.UnmatchedImpressionsMonitoring batch_size=1000 error_threshold=N/A is_jmx_monitoring=false start_time=2020-01-25_06:16:00 end_time=2020-01-25_06:27:45 duration=705808 read_count=18 write_count=18 filtered_count=0 error_count=0

I attached a screenshot of logs from when query starts until it fails. Please let me know if more details are needed. Any insights into this are much appreciated! alt text

Labels (2)
0 Karma

New Member

read_count is the number of row read by the input and write is the number of rows written to an index.

0 Karma

Explorer

What you are saying now is impossible...

2020-08-05 15:59:47.101 +0400 INFO c.s.dbx.server.task.listeners.JobMetricsListener - action=collect_job_metrics connection=XXX jdbc_url=null db_read_time=86 hec_upload_time=124 hec_record_process_time=195 format_hec_success_count=10380 hec_upload_bytes=9885449 status=COMPLETED input_name=XXXinput batch_size=1000 error_threshold=N/A is_jmx_monitoring=false start_time=2020-08-05_03:59:46 end_time=2020-08-05_03:59:47 duration=414 read_count=331755987 write_count=331599055 filtered_count=0 error_count=0

batch_size is 1000... How can i read 331755987 and write 331599055...

I´m not able to find the number of rows being read and written to the index apart from counting directly from the index and sourcetype...

0 Karma