I have a DB Connect connection to my DB that validates. The query that I send to the DB is displayed here:
WITH "dte" as (SELECT * FROM "T_AUDIT_LOG_HISTORY" UNION SELECT * FROM "T_AUDIT_LOG" ) select * from "dte" where "UN_ID" > ? ORDER BY "UN_ID" ASC
I use a rising value on column 10 ("UN_ID") which is a integer unique identifier that increases for every new record. This table is not updated. Only inserts arrive. The first column has a timestamp that I link to the _time internal field. What I would expect is that every unique id is imported just once, but this is not the case. Every 15 minutes it imports a full copy of the whole table...
Here is my config file for this connector:
[AUDIT_LOG_HIST] connection = Production disabled = 0 host = XXX_PROD index = xxx index_time_mode = dbColumn input_timestamp_column_number = 1 interval = */15 * * * * mode = rising query = WITH "dte" as (SELECT * \ FROM "T_AUDIT_LOG_HISTORY"\ UNION\ SELECT * \ FROM "T_AUDIT_LOG"\ )\ select *\ from "dte"\ where "UN_ID" > ?\ ORDER BY "UN_ID" ASC query_timeout = 60 sourcetype = audit:log tail_rising_column_number = 10
I would only need the new ids so I don't see any doubles in my index.
Thanks in advance