All Apps and Add-ons

How do I start a new Splunk DB Connect 2 input without getting all the rows initially?

mfscully
Explorer

I want to start a new DB Connect input for a table that has two months of data. I only want to grab the last day initially when I set up the new DB Connect input. Where do I set the initial rising setting?

0 Karma

igritsak
Engager

I just did the same process with DB Connect v2. I was migrating from DB Connect v1 (DBX) to the new app but already had 50K+ rows in Splunk.

From the Splunk docs here's the particular line:

A checkpoint value: The checkpoint value is how DB Connect determines what rows are new from one input execution to the next. The first time the input is run, DB Connect will only select those rows that contain a higher value in the checkpoint column than the checkpoint value you specify. Each time the input is finished running, DB Connect updates the input's checkpoint value with the value in the last row of the checkpoint column.

http://docs.splunk.com/Documentation/DBX/2.3.1/DeployDBX/Createandmanagedatabaseinputs

So in my case, I entered the highest value that Splunk already had indexed after I turned off the DB Connect v1 plugin.

0 Karma

Richfez
SplunkTrust
SplunkTrust

In DBX V2, there appears to be a "Checkpoint Value" in the section "Specify Rising Column" in the docs here. I don't have DBX2 available at the moment to confirm, but it seems likely that should do what you want.

You might also be able to use a custom SQL query - I think DB connect v2 does it like DB connect V1, so you could see the answer here on some ways to adjust the SQL to make it do what you want.

0 Karma

jkat54
SplunkTrust
SplunkTrust

It looks like there is a setting for MAX_ROWS or max_rows in one of the configuration files.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...