topic Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs in All Apps and Add-ons

Splunk DB Connect V3 - How create Automated / Programmatic creation of connections and inputs

gjanders — Tue, 09 Aug 2022 13:42:30 GMT

Please note that I'm answering this question in addition to posting it, if you find the answer useful please feel free to vote/award points!

The problem

After upgrading from Splunk DB Connect V2 to Splunk DB Connect V3 I was disappointed to find that some of the previously useful REST API's I was using to programatically create new connections/data inputs were no longer functional.

After logging a support ticket I was advised these were never supported, and therefore my request is now an "enhancement".

After quite a bit of work I did find a way to automate the DB Connect V3, it is likely unsupported by the official application but it works and provides a nice workaround until they officially support automation via REST API or similar.

Solution

Please see the answer below, comments/better solutions are welcome!

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Tue, 29 Sep 2020 13:28:46 GMT

The solution I have managed to use for connections/database inputs is below, improved solutions are welcome. If you are using DB Connect V2 you might want to try the REST API solution I found https://answers.splunk.com/answers/452618/how-do-i-use-the-restful-webservices-to-setup-new.html#answer-452619 note that this solution does not work for DB Connect V3!

EDIT: 2018-03-20 updated to include comments from ehudb and gsrivastava regarding a new REST endpoint in DB Connect 3.x

Identities

Quoting the answer in this thread from ehudb, use the REST endpoint:
https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/identities

Example

curl -k -X POST -u admin:changeit https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/identities -d "{\"name\":\"myuser\",\"username\":\"myuser\",\"password\":\"mypassword\"}"

Connections

The workaround solution of editing the db_connections.conf file and then trigger a get request to:
https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/configs/conf-db_connections/_reload

Is no longer required, as per the comments by gsrivastava you can use:
https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/connections

Database Inputs

Quoting/paraphrasing the comments from gsrivastava, the URL of:
https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/inputs

Can be used to create DB connect inputs

JSON data is required, for example:

{
"name": "ABCD",
"query": "select from ABCD",
"interval": "17 ",
"index": "test",
"mode": "rising",
"connection": "abcd",
"rising_column_index": 1,
"timestamp_column_index": 1,
"timestampType": "dbColumn",
"sourcetype": "abcd",
"checkpoint": {"value":"2018-03-22 00:00:00.000","appVersion":"3.1.1","columnType":93,"timestamp":"2018-03-22T11:06:11.000+05:30"}
}

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

jcoates_splunk — Wed, 29 Mar 2017 17:55:08 GMT

Cool. FWIW, we don't document this yet because it's not stable, as you've found. Glad it worked for you!

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Thu, 30 Mar 2017 00:17:59 GMT

Yes, I would like to see future verisons include automation by default as there is rarely a use case where we don't want some level of automation within an application!

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

saranya_fmr — Tue, 29 Sep 2020 15:11:25 GMT

Hi @Anonymous ,

I also wanted to automate the creation of DB Connection in DBX3 and I tried the below but it didnt seem to work:

1) Updated via CLI identities.conf and db_connections.conf with the new DB details
2) Did a reload on the host
curl -k -u admin:password https://myhost:8089/servicesNS/nobody/splunk_app_db_connect/configs/conf-db_connections/_reload

Is there anything I'm missing out? Could you please guide..

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Thu, 03 Aug 2017 00:40:06 GMT

This looks a bit different to what I have done:
https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/inputs/_reload

And:
https://localhost:8089/servicesNS/nobody/search/admin/server/_reload

Can you try those URL's? In particular the second one reloads the task server which should force it to reload the connections...

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

saranya_fmr — Tue, 29 Sep 2020 15:12:06 GMT

Hi @Anonymous ,

Did you create DB coonection and Identity via CLI initially and then did a reload via Splunk RESTAPI? What was your approach?

Im only creating a DB connection for DB Data Query and I do not want to create DB Outputs or inputs so reload inputs URL is not needed for me I think:

However I did try both the URLs that you suggested , reload task-server and reload inputs but it didnt work 😞
https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/inputs/_reload

I see the connection in UI but it gives me an error as seen in screenshot:

The splunk_app_db_connect_server.log shows below error:

2017-08-03 08:33:46.048 -0400 [dw-49 - GET /api/connections/SPOTFIRE60_DEV/status] ERROR io.dropwizard.jersey.errors.LoggingExceptionMapper - Error handling a request: 94204c89a78800be
java.lang.IllegalArgumentException: Illegal base64 character 5f
at java.util.Base64$Decoder.decode0(Base64.java:714)
at java.util.Base64$Decoder.decode(Base64.java:526)
at java.util.Base64$Decoder.decode(Base64.java:549)
at com.splunk.dbx.crypto.AESUtils.decrypt(AESUtils.java:75)

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Fri, 04 Aug 2017 00:18:41 GMT

If the connection does not work in the UI then the REST API will not help here, I would suggest trying to get it working manually first...

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

saranya_fmr — Fri, 04 Aug 2017 11:22:58 GMT

Hi @garethatiag ,

I noticed that the password is not getting encrypted even after Splunk restart was the root cause.
How does the password get encrypted if restart is not the solution?

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Sat, 05 Aug 2017 17:44:50 GMT

I'm using the URL:
https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/identities/

And posting in the values for name/username/password and that takes care of any password encryption for me...

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

saranya_fmr — Mon, 07 Aug 2017 09:01:06 GMT

Hi @garethatiag ,

This URL doesnt update password/username for DB Connect V3. I assume it works only for DBX V2:

https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/identities/

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Tue, 29 Sep 2020 15:15:23 GMT

@saranya_fmr Unfortunately you are correct, I think I used that url for DBX 2 and now when I re-test with DBX 3 I cannot get that to change, I tested:
/servicesNS/nobody/splunk_app_db_connect/properties/identities
And also:
/servicesNS/nobody/splunk_app_db_connect/db_connect/identities/

And neither work...perhaps an enhancement request to the DB connect team?
It is frustrating that it cannot be automated...

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

sloshburch — Wed, 09 Aug 2017 12:39:54 GMT

@saranya_fmr - Something just crossed my mind that is tangential but could be extremely important for you: If you make the call with the REST API, won't the URI request appear in the _internal logs? Therefore, the username and password from the call would appear in clear text on the filesystem in the respective Splunk log AND therefore in the _internal index?

If we're lucky, those parameters are hashed or obfuscated as part of the call and don't appear but since you're not using splunkweb to make the call (and therefore no javascript to do that for you) it's possible they are not secure.

I would suggest validating that because you'll wanna make sure there's a usage approach that is secure before getting too far into implementing this way.

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gsrivastava — Wed, 21 Feb 2018 12:48:30 GMT

@garethatiag, As far as I know we cannot create a new identity by editing the identities.conf file as the password is stored in encrypted format and the encryption key is not known.

If identity cannot be created, how can we create a new connection by editing the db_connections.conf file?

So by your solution, Can we create a new connection with a new identity?

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Tue, 29 Sep 2020 18:11:09 GMT

When I first automated this I could use:
/servicesNS/nobody/splunk_app_db_connect/db_connect/identities/

So https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/identities/ or similar with a POST request and that would have created the username/password in the appropriate encrypted format. Unfortunately newer versions (3.x) of this application removed that REST endpoint and I'm unsure if they have allowed an equivalent one so far.

When I last checked I could not find the REST API to create identities, I'm hoping the DB Connect team intend to re-add or at least allow a programmatic way of doing this...perhaps send them an enhancement request via the support portal?

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

wlaenig — Mon, 05 Mar 2018 21:13:04 GMT

i believe identity.dat is in the certs directory is the encryption key with aes-256. however i have not able to decrypt the encrypted password.

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

ehudb — Mon, 19 Mar 2018 08:57:23 GMT

You can add new Identities with encryption in DB Connect 3.x:
Use the REST endpoint:
https://splunkserver:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/identities

Example:
curl -k -X POST -u admin:password https://splunkserver:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/identities -d "{\"name\":\"myuser\",\"username\":\"myuser\",\"password\":\"mypassword\"}"

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gjanders — Tue, 20 Mar 2018 02:02:48 GMT

That works, thankyou!

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gsrivastava — Tue, 20 Mar 2018 05:46:13 GMT

Its working fine, thank you!

Re: Splunk DB Connect V3 - Automated / Programmatic creation of connections and inputs

gsrivastava — Tue, 20 Mar 2018 06:13:14 GMT

Now a similar endpoint is also available for creating connections -
https://splunkserver:8089/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/connections

We do not need to use a workaround of editing the db_connections.conf file and then triggering reloads to the server.