All Apps and Add-ons

Splunk DB Connect 3: Splunk skips some data when using epochtime as rising column

Gong1027
Explorer

Hi all,
we've been using Splunk DB Connect 3.0.2 on Splunk 6.5.2 for ingesting data from Oracle database.
and I chose 'Rising Column' as the Input Type and set an epochtime field as the Checkpoint Column.
and the ingestion runs on hourly basis.

however, I found it keeps skipping records especially when records reached in DB at almost the same time.
could you shed some light on how this can happen and what shall be the appropriate setting if I need to use epochtime as rising field?

Thanks,
Gong

Tags (1)
0 Karma

mstjohn_splunk
Splunk Employee
Splunk Employee

Hi @Gong1027. Were you able to test out @jcoates ’s solution? Did it work? If yes, please don't forget to resolve this post by clicking on "Accept". If you still need more help, please provide a comment with some feedback. Thanks and happy Splunking!

0 Karma

jcoates
Communicator

Hi, if you scroll down to Rising Input here, there’s some discussion of the problem: http://docs.splunk.com/Documentation/DBX/3.1.3/DeployDBX/Createandmanagedatabaseinputs

You’re most likely seeing records come in at higher resolution than DBX can handle. In other words, if you get 5 records per second and the checkpoint is at a 1 second interval, then you’ll lose 4 records from each execution.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...