Monitoring Splunk

Splunk DB Connect - How to reset tail.rising state?

mewtwo
Explorer

How to reset tail.rising state that go back and read everything from the beginning of my database table?

Tags (2)
1 Solution

ziegfried
Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

View solution in original post

ziegfried
Influencer

The persistent state of database inputs is stored in $SPLUNK_DB/persistentstorage/dbx ($SPLUNK_DB is, if not specified otherwise: $SPLUNK_HOME/var/lib/splunk). Each input has it's own directory, which is a hash of it's name (ie. a 32 character long hex string). This directory typically contains 2 files:

  • manifest.properties: contains meta-information, such as the name of the input
  • state.xml: contains the actual state in XML format

So you need to first identify the state directory and then you can modify (or delete) the XML file.

For modifying it, this state file looks something like this:

<list>
  <value key="latest.record_update">
    <value class="sql-timestamp">2012-12-07 04:22:25.703</value>
  </value>
</list>

The value may vary, depending on the datatype of the rising-column you chose.

A Splunk restart might be necessary in order for DB Connect to recognize the changes.

Glenn
Builder

FYI neither Splunk or DB connect restart is required for an updated state.xml to take effect with at least the combination of Splunk 6.2.3 and DB Connect 1.2.0 (tested).

0 Karma

usd0872
Path Finder

I second that for splunk 6.3.3 with DB Connect 1.1.7.
Disable the input; change the file; enable the input. Works fine.

0 Karma

harshavrath
Contributor

Hi guys, I'm facing problems with dbx my Java Bridge is not starting so by reading few answers i got to know it might be the case that the state.xml be corrupted,when i checked the state.xml file its empty.So what I'm i supposed to include in the state.xml

0 Karma

Sriram
Communicator

which process is locking the state.xml and manifest file. It appears to be sporadic as well. I have the same issue. Unable to delete or modify the file to reset the state. I ended up cloning the db input and disable the old one. Appreciate your help.

0 Karma

baerrach
Path Finder
0 Karma

rettops
Path Finder

Note that deleting the database input does not remove or reset the persistent state. If you want to start over, you need to delete the input, then delete the persistent state directory, then create the input again.

0 Karma

piebob
Splunk Employee
Splunk Employee

ziggy, let's work on getting a procedure for the main use case (reading in only the most recent entries) into the docs.

0 Karma

bigtyma
Communicator

Suggestions if the files are locked? I have stopped splunkd and I am unable to modify the files.

mewtwo
Explorer

Danke, das ist sehr toll Applikation!

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...