Splunk Search

Configuring dbx tail input on very large table. (70,000,000+ rows)

johnebgood
Path Finder

I was able to get the input to work but it would be nice to start the tail from a specified value when the input is configured. I don't want to input all 70,000,000 rows on the first run... I just want the data from the last day or so as I would blow up our license since I have to do this for 20+ databases with similarly sized tables. Any ideas on how to do this?

Tags (2)
1 Solution

hirsts
Path Finder

Creating the monitor with initial SQL to return one row only, updating the state.xml and then remove the 1 row limit worked fine. I did not need to restart Splunk as the state.xml file is read each time the monitor is run.

hirsts
Path Finder

I've tied this and the state.xml file is only created after the first run of the monitor. In my the case the first run returned lots of rows which then seem to get batch inserting into the indexing queues as the state.xml files keeps changing until the backlog is indexed.

I'm going to try creating the monitor with SQL that only returns 1 row at a time, disable the monitor, modify the state.xml, modify the SQL to return all the required rows then re-enable the monitor.

0 Karma

johnebgood
Path Finder

Note, after you create the new database monitor immediately disable it and go and edit the state.xml file. I have been copying the state.xml from previously configured monitors, makes it fairly easy.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...