Monitoring Splunk

Splunk DB Connect - How to reset tail.rising state?

mewtwo
Explorer

How to reset tail.rising state that go back and read everything from the beginning of my database table?

Tags (2)
1 Solution

ziegfried
Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

View solution in original post

ziegfried
Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

Glenn
Builder

FYI neither Splunk or DB connect restart is required for an updated state.xml to take effect with at least the combination of Splunk 6.2.3 and DB Connect 1.2.0 (tested).

0 Karma

usd0872
Path Finder

I second that for splunk 6.3.3 with DB Connect 1.1.7.
Disable the input; change the file; enable the input. Works fine.

0 Karma

harshavrath
Contributor

Hi guys, I'm facing problems with dbx my Java Bridge is not starting so by reading few answers i got to know it might be the case that the state.xml be corrupted,when i checked the state.xml file its empty.So what I'm i supposed to include in the state.xml

0 Karma

Sriram
Communicator

which process is locking the state.xml and manifest file. It appears to be sporadic as well. I have the same issue. Unable to delete or modify the file to reset the state. I ended up cloning the db input and disable the old one. Appreciate your help.

0 Karma

baerrach
Path Finder
0 Karma

rettops
Path Finder

Note that deleting the database input does not remove or reset the persistent state. If you want to start over, you need to delete the input, then delete the persistent state directory, then create the input again.

0 Karma

piebob
Splunk Employee
Splunk Employee

ziggy, let's work on getting a procedure for the main use case (reading in only the most recent entries) into the docs.

0 Karma

bigtyma
Communicator

Suggestions if the files are locked? I have stopped splunkd and I am unable to modify the files.

mewtwo
Explorer

Danke, das ist sehr toll Applikation!

Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...