Splunk DB Connect 3: Splunk skips some data when using epochtime as rising column

[Gong1027]

Hi all,
we've been using Splunk DB Connect 3.0.2 on Splunk 6.5.2 for ingesting data from Oracle database.
and I chose 'Rising Column' as the Input Type and set an epochtime field as the Checkpoint Column.
and the ingestion runs on hourly basis.

however, I found it keeps skipping records especially when records reached in DB at almost the same time.
could you shed some light on how this can happen and what shall be the appropriate setting if I need to use epochtime as rising field?

Thanks,
Gong

[mstjohn_splunk]

Hi @Gong1027. Were you able to test out @jcoates ’s solution? Did it work? If yes, please don't forget to resolve this post by clicking on "Accept". If you still need more help, please provide a comment with some feedback. Thanks and happy Splunking!

[jcoates]

Hi, if you scroll down to Rising Input here, there’s some discussion of the problem: http://docs.splunk.com/Documentation/DBX/3.1.3/DeployDBX/Createandmanagedatabaseinputs

You’re most likely seeing records come in at higher resolution than DBX can handle. In other words, if you get 5 records per second and the checkpoint is at a 1 second interval, then you’ll lose 4 records from each execution.