DB connect - rising column trouble

[cguimezanes]

Hello,

I'm using a dbmon-tail to index a table of my database.
My rising column is a modification date (SQL Server DateTIme). My SQL request is a simple select with a {{WHERE $rising_column$ > ?}}.

In my table I have a primary key nammed "ID", a field "status" and my modification date nammed "updated_date". I have an other field "filename" not indexed by Splunk.

Sometimes in my results I have a duplication of the result for exemple:
Note: Every records start with a status "STEP_1"

In the database i have:
ID STATUS UPDATED_DATE
1 STEP_1 2015/03/22
2 STEP_2 2015/03/23

In splunk I have:
ID STATUS UPDATED_DATE
1 STEP_1 2015/03/22
2 STEP_1 2015/03/22 ==> Result not upated
2 STEP_2 2015/03/23 ==> But duplicated

Do you how splunk dbmon-tail does not update the record instead of duplicate the record?

Thks!

[vganjare · Accepted solution]

Hi,

Splunk DBConnect app writes the records into splunk index. Once the record is written into splunk index, there is no way to change that record. If there are any modifications in the DB records, and if the rising column is configured for "updated_time" column, then DBConnect will look for all the records which are added/updated after last DB fetch. The updated records will get picked up and will be indexed in splunk.

By using dedup command, duplicate records can be filtered out.

Thanks!

[cguimezanes]

Dedup works well. Thks