All Apps and Add-ons

DB Connect: With my Batch Input settings, will splunk keep pulling in the same rows or new ones until all rows are pulled in?

TBeekley
New Member

I'd like to bring in about 26 million events into Splunk using DB Connect and I was thinking Batch Input would be best to start with. It's for historical purposes and will only be done once and we'll move to rising column after the first index.

If Max Rows to Retrieve = 1000000 and Fetch Size = 10000, does Splunk keep pulling in the same rows? Or does it pick up new ones until all of the rows are pulled in?

0 Karma

davpx
Communicator

Why not just start with rising column? If you run it as a batch, it will run again on your configured interval and re-index all the records again if you do not disable the input before the 2nd run.

Max rows to retrieve limits the total amount of records pulled for a batch input, the default is 0. Fetch size is how many records dbconnect will pull per pass to keep from requesting 26 million rows at once.

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...