All Apps and Add-ons

DB connect 3.5.1 missing data from Oracle database

dailv1808
Path Finder

Hi Friends,

I have query input in DB connect app, which runs collect data from oracle database.

Database generates 800 records per second .

I'm using "Rising mode" with SEQUENSE_NO as Rising column. SEQUENSE_NO is unique and ascending column.

My query look like below:

SELECT * FROM table

WHERE SEQUENSE_NO > ? AND tnx_stamp > sysdate - 10/1440

ORDER BY SEQUENSE_NO ASC

My Input setting:

Max row to Retrieve: 0 (Unlimited )

Fetch Size: 1000 Or 100000

Execution Frequency: 120s

This input work fine, no Error Or waring message in _internal log.

Next, I compared number of record between Oracle and Splunk in a time range. Its difference.

In Oracle 140,425 records, Splunk less records than Oracle with 135,008 events (I have tried with many different range of time, it same result.😞

dailv1808_0-1624349501600.pngdailv1808_1-1624349675131.png

 

Next, I search count timechart span=1s. --> Data loss occurs periodically

dailv1808_4-1624351368274.png

Next step. I checked internal dbx_job_metrict log.  The time of data loss coincides with the job start_time.

I dont know why it happened.

dailv1808_5-1624351721302.png

 

Would appreciate any help figuring out how to resolve this. Thanks!

 

Labels (1)
0 Karma

dailv1808
Path Finder

@splunkcan you help me?

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!