I am using DB Connect 2.3.1. I am successfully importing data from a number of MS SQL databases already, and want to add a new database input.
The data in this new database goes back to early 2008. Unfortunately the primary key is not a bigint; it is a guid (char), so instead of using that as the rising column I have used a datetime column. When I attempted to import all data starting from checkpoint value 2008-01-01 it brought back a load of rows (~55k), running through to the current date, but definitely not as many as there are in the database (~880k).
Other settings for the input were:
Max Rows to Retrieve = 10,000
Fetch Size = left blank, apparently defaults to 300
Timestamp Column is the same datetime column used as the Rising Column
Execution Frequency = 120 seconds
Can anyone explain why I don't have as much data as expected, yet the checkpoint value is up-to-date?
Query
Here is an anonymized version of my query, entered in Advanced mode. For operational reasons I need to "buffer" the input by 6 hours to ensure events have been written to the database from distributed sources:
SELECT DateTime, ColA, ColB, ColC, ColD
FROM Table1
WHERE DateTime < DATEADD(hh, -6, GETDATE())
AND DateTime > CAST(? AS datetime)
... View more