Hi Friends,
I have query input in DB connect app, which runs collect data from oracle database.
Database generates 800 records per second .
I'm using "Rising mode" with SEQUENSE_NO as Rising column. SEQUENSE_NO is unique and ascending column.
My query look like below:
SELECT * FROM table
WHERE SEQUENSE_NO > ? AND tnx_stamp > sysdate - 10/1440
ORDER BY SEQUENSE_NO ASC
My Input setting:
Max row to Retrieve: 0 (Unlimited )
Fetch Size: 1000 Or 100000
Execution Frequency: 120s
This input work fine, no Error Or waring message in _internal log.
Next, I compared number of record between Oracle and Splunk in a time range. Its difference.
In Oracle 140,425 records, Splunk less records than Oracle with 135,008 events (I have tried with many different range of time, it same result.😞
dailv1808_0-1624349501600.png
dailv1808_1-1624349675131.png
Next, I search count timechart span=1s. --> Data loss occurs periodically
dailv1808_4-1624351368274.pngNext step. I checked internal dbx_job_metrict log. The time of data loss coincides with the job start_time.
I dont know why it happened.
dailv1808_5-1624351721302.png
Would appreciate any help figuring out how to resolve this. Thanks!
