topic DB Connect: With my Batch Input settings, will splunk keep pulling in the same rows or new ones until all rows are pulled in? in All Apps and Add-ons

DB Connect: With my Batch Input settings, will splunk keep pulling in the same rows or new ones until all rows are pulled in?

TBeekley — Wed, 31 Jan 2018 15:15:16 GMT

I'd like to bring in about 26 million events into Splunk using DB Connect and I was thinking Batch Input would be best to start with. It's for historical purposes and will only be done once and we'll move to rising column after the first index.

If Max Rows to Retrieve = 1000000 and Fetch Size = 10000, does Splunk keep pulling in the same rows? Or does it pick up new ones until all of the rows are pulled in?

Re: DB Connect: With my Batch Input settings, will splunk keep pulling in the same rows or new ones until all rows are pulled in?

davpx — Wed, 31 Jan 2018 23:32:46 GMT

Why not just start with rising column? If you run it as a batch, it will run again on your configured interval and re-index all the records again if you do not disable the input before the 2nd run.

Max rows to retrieve limits the total amount of records pulled for a batch input, the default is 0. Fetch size is how many records dbconnect will pull per pass to keep from requesting 26 million rows at once.