All Apps and Add-ons

Splunk DB Connect 2: How to only collect new rows since the last update?

arjangoos
Path Finder

In DB Connect, I only want to collect the new rows since the last update. Now each time it collects the last 1000 records because that is the Max Rows to Retrieve.

Tags (1)
0 Karma

woodcock
Esteemed Legend

This is generally done by using Rising Column. I see that you have configured BOTH tail_rising_column_name and tail_rising_column_number which may be your problem (if they are not the same and splunk is giving the wrong one precedence). I would delete one of those settings and make sure that the other one is correctly pointing to a value that rises with time (such as an epoch timestamp).

0 Karma

arjangoos
Path Finder

this is my inputs.conf

[mi_input://opentunnel_acc_metrics]
connection = opentunnel_acc
index = acc_opentunnel_db
interval = 150
max_rows = 10
mode = tail
output_timestamp_format = epoch
query = SELECT * FROM "OPENTUNNEL_RUNTIME"."METRICS"
source = /opt/splunk/var/log/splunk/acc_opentunnel_db.log
sourcetype = acc_opentunnel_db
tail_follow_only = 1
tail_rising_column_name = TUNNEL_ENTRY_TS
tail_rising_column_number = 1
ui_query_catalog = NULL
ui_query_mode = advanced
ui_query_schema = OPENTUNNEL_RUNTIME
ui_query_table = METRICS
tail_rising_column_checkpoint_value = 1443682436344
input_timestamp_column_name = TUNNEL_ENTRY_TS
input_timestamp_column_number = 17

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...