Splunk Enterprise

Storage for DB Connect Checkpoint values

altink
Builder

Hi

Where are the Checkpoint values for enabled DB Connect Inputs stored?

I did check at folder:
/opt/splunk/var/lib/splunk/modinputs/server/splunk_app_db_connect

There there are only files with names of our disabled DB Inputs, but not the ones of our enabled DB Inputs.

Splunk Enterprise Version: 9.0.4.1
Splunk DB Connect Version: 3.6.0

Ps. our three enabled DB Inputs do work correctly, and I can see the checkpoint values from the web.
Just cannot find where they are stored on the OS

best regards
Altin

Labels (1)
0 Karma

altink
Builder

Upgrade is not an option for now. furthermore, everything goes OK with DB Connect. it will be upgraded with the whole system.

I did ask this question precisely because could not find the checkpoint values persistence files for our enable DB Input in folder:
/opt/splunk/var/lib/splunk/modinputs/server/splunk_app_db_connect

0 Karma

altink
Builder

Thank You @Richfez 

As a first need (and I should have said this in the opening), I was not asking to access them at all. I just want to know where they are, to backup them just like everything else in /etc/apps folder.

But editing them could also be a need, ex: in case of db inputs info  loss of any kind.  And if it be the case, I guess to better edit them (at one's own risk) directly into the OS via a text editor since they are json.

regards
Altin


0 Karma

Richfez
SplunkTrust
SplunkTrust

Ah, backups.  Splunk has this documented, so I'll just point you to their docs on "Backup and restore Splunk DB Connect version 3.10.0 or higher"

Hope that helps!

-Rich

0 Karma

altink
Builder

The Documentation link given belongs to DB Connect version 3.15.0 and talks for "Splunk DB Connect version 3.10.0 or higher".

Our DB Connect Version is: 3.6.0

When on the doc link I change the DB Connect version to 3.6.0, I receive this:

"The topic you've asked to see does not apply to the version you selected. "

So it looks documentation tells nothing for backup/restore of DB Connect 3.6.0.

Or am I missing something?

best regards
Altin

0 Karma

Richfez
SplunkTrust
SplunkTrust

3.6 is old and you should update, then the current documentation would work.  🙂

But, it also had a simpler system of checkpoints, storing those in files on disk.

I cannot remember exactly where those were, but maybe something like $SPLUNK_HOME/var/lib/splunk/modinputs/dbx_input...?  I don't know, I have that directory and it seems familiar, but it's empty on my system because I try to keep up to date.  (Hehehe, you knew I was going to say that, didn't you... ?  🙂

 

0 Karma

Richfez
SplunkTrust
SplunkTrust

BTW I don't know if it's clear, but

a) you should be able to find the checkpoint files on disk, but ..

b) even if you don't, if you back up $SPLUNK_HOME/var/lib/modinputs I think you effectively back up your checkpoint files. 

A bit of interwebs searching ought to confirm this.

Also note that the checkpoint files are useless if you are trying to back them up pre-updating (at least if you cross the magical version near 3.10 where it switches from checkpoint files to KV store entries), because you can't slap them into place and expect it to find/use them any more.  It should migrate them during the upgrade, but I'm not sure it'll ever "re-migrate" later if you have to try to restore files into a kvstore based system.  YMMV, etc.

0 Karma

Richfez
SplunkTrust
SplunkTrust

It's terrible, they're not easily accessible except through the UI.  It's a big ... sore spot for some of us who need to use these in a more programmatic way.

But, there is a way using the REST interface from cURL.

curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input

Obviously fix the username and password to an admin one, and your hostname if it's not on localhost.

 You might want to pipe that through jq to 'pretty print' it if you have jq installed because otherwise it's all smashed together and hard to read:

curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input | jq .

 You can also see only an individual one if you append the _key's value for the one you want to the end.  (The _key comes from the output of one of the earlier commands.)

curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input/6452ce6e55102d0ad735ec31 | jq .

You can also delete them or edit them, though ... obviously be careful and do this in a test environment at first!

curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input/6452ce6e55102d0ad735ec31 -X DELETE

And I've not found a good way to "edit" them, but it's pretty trivial to just edit the JSON you get from an individual entry, and load that back in wholesale.

curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input -d '{ "inputName" : "newEntryforMyDB", "value" : "200", "appVersion" : "3.16.0", "columnType" : 4, "timestamp" : "2024-03-21T13:11:41.633-05:00", "_user" : "nobody", "_key" : "65fc6ce1764e95450b0d98e1" }' -H "Content-Type: application/json"

Which would overwrite entry 65fc6... with that new information.

Happy Splunking,

Rich

 

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...