Hello,
I have a situation where I have an input that is losing some of the rows. The connection works stable. The input is executed every 5 minutes and defined with rising column on “LASTMODIFIED” (DateTime) column with ">=?" to avoid lost rows, order ASC. Vast majority of rows is indexed correctly but few are not. Interestingly when I use SQL Explorer I retrieve the rows without any problem. I have created special input that was 100% the same as the original one + had “and ID=123” in SQL query limiting the ingest to the one of the interesting rows only. It worked like charm. Answering question before asked – I cannot use any different column to be rising one (hence “>=” not the typical “>” in SQL query). Moreover, I have checked and found out that the missing “123 row” has no twin row with the same “LAST MODIFIED” value.
1. I am looking for ways of troubleshooting it. I tried to increase logging to DEBUG but I drown in the amount of logs and was unable to find anything that would be helpful in the obvious way – I was hoping for results of each row parsing, no luck…
2. If anyone can suggest possible causes that are not covered by “>=” trick I would also truly appreciate.
Thank you in advance 🙂
