All Apps and Add-ons

Splunk DB Connect 2: How to only collect new rows since the last update?

arjangoos
Path Finder

In DB Connect, I only want to collect the new rows since the last update. Now each time it collects the last 1000 records because that is the Max Rows to Retrieve.

Tags (1)
0 Karma

woodcock
Esteemed Legend

This is generally done by using Rising Column. I see that you have configured BOTH tail_rising_column_name and tail_rising_column_number which may be your problem (if they are not the same and splunk is giving the wrong one precedence). I would delete one of those settings and make sure that the other one is correctly pointing to a value that rises with time (such as an epoch timestamp).

0 Karma

arjangoos
Path Finder

this is my inputs.conf

[mi_input://opentunnel_acc_metrics]
connection = opentunnel_acc
index = acc_opentunnel_db
interval = 150
max_rows = 10
mode = tail
output_timestamp_format = epoch
query = SELECT * FROM "OPENTUNNEL_RUNTIME"."METRICS"
source = /opt/splunk/var/log/splunk/acc_opentunnel_db.log
sourcetype = acc_opentunnel_db
tail_follow_only = 1
tail_rising_column_name = TUNNEL_ENTRY_TS
tail_rising_column_number = 1
ui_query_catalog = NULL
ui_query_mode = advanced
ui_query_schema = OPENTUNNEL_RUNTIME
ui_query_table = METRICS
tail_rising_column_checkpoint_value = 1443682436344
input_timestamp_column_name = TUNNEL_ENTRY_TS
input_timestamp_column_number = 17

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...