Archive

Splunk DB Connect - How to reset tail.rising state?

Explorer

How to reset tail.rising state that go back and read everything from the beginning of my database table?

Tags (2)
1 Solution

Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

View solution in original post

Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

View solution in original post

Builder

FYI neither Splunk or DB connect restart is required for an updated state.xml to take effect with at least the combination of Splunk 6.2.3 and DB Connect 1.2.0 (tested).

0 Karma

Path Finder

I second that for splunk 6.3.3 with DB Connect 1.1.7.
Disable the input; change the file; enable the input. Works fine.

0 Karma

Contributor

Hi guys, I'm facing problems with dbx my Java Bridge is not starting so by reading few answers i got to know it might be the case that the state.xml be corrupted,when i checked the state.xml file its empty.So what I'm i supposed to include in the state.xml

0 Karma

Communicator

which process is locking the state.xml and manifest file. It appears to be sporadic as well. I have the same issue. Unable to delete or modify the file to reset the state. I ended up cloning the db input and disable the old one. Appreciate your help.

0 Karma

Path Finder
0 Karma

Path Finder

Note that deleting the database input does not remove or reset the persistent state. If you want to start over, you need to delete the input, then delete the persistent state directory, then create the input again.

0 Karma

Splunk Employee
Splunk Employee

ziggy, let's work on getting a procedure for the main use case (reading in only the most recent entries) into the docs.

0 Karma

Communicator

Suggestions if the files are locked? I have stopped splunkd and I am unable to modify the files.

Explorer

Danke, das ist sehr toll Applikation!