All Apps and Add-ons

DB Connect: With my Batch Input settings, will splunk keep pulling in the same rows or new ones until all rows are pulled in?

TBeekley
New Member

I'd like to bring in about 26 million events into Splunk using DB Connect and I was thinking Batch Input would be best to start with. It's for historical purposes and will only be done once and we'll move to rising column after the first index.

If Max Rows to Retrieve = 1000000 and Fetch Size = 10000, does Splunk keep pulling in the same rows? Or does it pick up new ones until all of the rows are pulled in?

0 Karma

davpx
Communicator

Why not just start with rising column? If you run it as a batch, it will run again on your configured interval and re-index all the records again if you do not disable the input before the 2nd run.

Max rows to retrieve limits the total amount of records pulled for a batch input, the default is 0. Fetch size is how many records dbconnect will pull per pass to keep from requesting 26 million rows at once.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...