Monitoring Splunk

Splunk DB Connect - How to reset tail.rising state?

mewtwo
Explorer

How to reset tail.rising state that go back and read everything from the beginning of my database table?

Tags (2)
1 Solution

ziegfried
Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

View solution in original post

ziegfried
Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

Glenn
Builder

FYI neither Splunk or DB connect restart is required for an updated state.xml to take effect with at least the combination of Splunk 6.2.3 and DB Connect 1.2.0 (tested).

0 Karma

usd0872
Path Finder

I second that for splunk 6.3.3 with DB Connect 1.1.7.
Disable the input; change the file; enable the input. Works fine.

0 Karma

harshavrath
Contributor

Hi guys, I'm facing problems with dbx my Java Bridge is not starting so by reading few answers i got to know it might be the case that the state.xml be corrupted,when i checked the state.xml file its empty.So what I'm i supposed to include in the state.xml

0 Karma

Sriram
Communicator

which process is locking the state.xml and manifest file. It appears to be sporadic as well. I have the same issue. Unable to delete or modify the file to reset the state. I ended up cloning the db input and disable the old one. Appreciate your help.

0 Karma

baerrach
Path Finder
0 Karma

rettops
Path Finder

Note that deleting the database input does not remove or reset the persistent state. If you want to start over, you need to delete the input, then delete the persistent state directory, then create the input again.

0 Karma

piebob
Splunk Employee
Splunk Employee

ziggy, let's work on getting a procedure for the main use case (reading in only the most recent entries) into the docs.

0 Karma

bigtyma
Communicator

Suggestions if the files are locked? I have stopped splunkd and I am unable to modify the files.

mewtwo
Explorer

Danke, das ist sehr toll Applikation!

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

How to Break a Kubernetes Pod on Purpose and Let the AI Troubleshooting Agent Find ...

Let's say I've just rolled out a Kubernetes Deployment with a typo in the image tag. Nothing explodes. The pod ...

Splunk Add-on for Microsoft Windows 11.0.3: Parse Once, Search Faster

The updated Splunk Add-on for Microsoft Windows helps customers search Windows events faster by reducing ...

What's New in Splunk Enterprise Security 8.8 and Beyond

Threats move at machine speed. Keeping up should not depend on having the biggest budget, the largest team, or ...