topic Re: Consume API of a remote hosted splunk in Getting Data In

Consume API of a remote hosted splunk

tbhasme — Wed, 30 Sep 2020 01:14:32 GMT

I have a react app running locally and I need to consume APIs of Splunk which is hosted on some other server. In order to talk to the that server, here are the things I did:
1. Updated splunk with my own signed certificates and installed the CA vert on my machine and browser (as listed on https://docs.splunk.com/Documentation/Splunk/7.3.0/Security/Howtoself-signcertificates, https://docs.splunk.com/Documentation/Splunk/7.3.0/Security/HowtoprepareyoursignedcertificatesforSplunk, and https://docs.splunk.com/Documentation/Splunk/7.3.0/Security/ConfigureSplunkforwardingtousesignedcertificates)
2. set crossOriginSharingPolicy = *

Here is my client code:

var details = {
            'username': 'admin',
            'password': '<pwd>'
        };
        var formBody = [];
        for (var property in details) {
          var encodedKey = encodeURIComponent(property);
          var encodedValue = encodeURIComponent(details[property]);
          formBody.push(encodedKey + "=" + encodedValue);
        }
        formBody = formBody.join("&");
        console.log(formBody)
        fetch("https://localhost:8089/services/auth/login", {
              method: "POST",
              body: formBody,                  
              headers: { 'Content-type': 'application/x-www-form-urlencoded',
             }

            })
              .then(res => {
                if (res.ok) {
                    console.log("Response json data. -->" + res)
                  return res;
                } else {
                  throw Error(res.statusText);
                }
              })
              .then(json => {
                this.setState({
                  token: json
                });
              })
              .catch(error => console.error(error));

Splunk is hosted on a remote server which I am accessing through port forwarding.

With this setup,
1. On Chrome: I am getting ERR_CERT_COMMON_NAME_INVALID. It seems like CN mismatch but I really don't think that is the issue.
2. On FIrefox: I can see the session token in web console but in code, i get a response object of type CORS.

I am not really sure where have I gone wrong.

Re: Consume API of a remote hosted splunk

jkat54 — Thu, 04 Jul 2019 20:17:17 GMT

Of course that's the issue. You're proxying through localhost to get to a remote sever with a different CN.

You could disable ssl cert validation in your app or directly connect to the server using its proper CN/FQDN

Re: Consume API of a remote hosted splunk

tbhasme — Fri, 05 Jul 2019 10:22:28 GMT

I have also tried by disabling certificate validation in Chrome but that result in an empty response of type cors.

Re: Consume API of a remote hosted splunk

jkat54 — Fri, 05 Jul 2019 13:01:58 GMT

It's just not going to work this way. Either fetch the splunk endpoint by its fqdn/cn (and be sure it's CA is in your trusted root store) or find another way to do what you want.

Re: Consume API of a remote hosted splunk

tbhasme — Wed, 30 Sep 2020 01:11:14 GMT

It is not possible to access splunk using its fqdn as it is in some other network only accessible through ssh.
I have already configured Splunk with a CA whose CN I have kept '*.local'. The server certificate is also generated with same CN. That CA is also installed in my machine as well as Chrome. I am then running my local app as http://splunk.local:3000 which hits splunk on https://splunk.local:8089.
It still says ERR_CERT_COMMON_NAME_INVALID.

However, if this is a cert issue, it should have worked with ssl disabled. When I tried by disabling splunkd's ssl, I again started getting response type of cors.

Re: Consume API of a remote hosted splunk

jkat54 — Fri, 05 Jul 2019 18:42:04 GMT

Have you seen the "hosts" file before?

Put the proper IP and fqdn in your hosts file and enjoy.

Another issue might be that the cert you think you have installed on the splunk server is not the correct one. Try using

openssl s_client -connect yourserver:8089

And share the print out.

Re: Consume API of a remote hosted splunk

tbhasme — Fri, 05 Jul 2019 18:48:16 GMT

Yes, I have already made the entry (127.0.0.1 splunk.local). That's how I am accessing splunk.local.

Here is the output:

 openssl s_client -connect splunk.local:8089
CONNECTED(00000003)
depth=0 C = AU, ST = Some-State, O = Splunk, CN = *.local
verify error:num=18:self signed certificate
verify return:1
depth=0 C = AU, ST = Some-State, O = Splunk, CN = *.local
verify return:1
---
Certificate chain
 0 s:/C=AU/ST=Some-State/O=Splunk/CN=*.local
   i:/C=AU/ST=Some-State/O=Splunk/CN=*.local
 1 s:/C=AU/ST=Some-State/O=Splunk/CN=*.local
   i:/C=AU/ST=Some-State/O=Splunk/CN=*.local
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDBjCCAe4CCQDgMbcsg7UyGTANBgkqhkiG9w0BAQsFADBFMQswCQYDVQQGEwJB
VTETMBEGA1UECAwKU29tZS1TdGF0ZTEPMA0GA1UECgwGU3BsdW5rMRAwDgYDVQQD
DAcqLmxvY2FsMB4XDTE5MDcwNTE3MTkwMVoXDTIyMDcwNDE3MTkwMVowRTELMAkG
A1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxDzANBgNVBAoMBlNwbHVuazEQ
MA4GA1UEAwwHKi5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AM0j+3TbxoOwbHhDc6gF/stgNnUr3IAbZ8Yw2UJued32lrtHD6yGq9ESQ6vBNbFC
AwUpxoItc9/EX7b6Ct7uJOpWVNOd54CosuVkZksqIAyrI0lZ7OLqkH2yUseSWCIA
GRkCfxSTzmr3oPyId/ifuWvQcdMPDF4Af4XNLycSUo3vv+RZXf8oTnJY4+0vKXuy
JonxsilcKhVkUAOmyH7U9rK1Z041Hnlsx7FHIwmcf5U4ThLtyJmSOHcpH/ZA8gxt
9OrxaDV6RnltSzXfrE9E+QXcn7K7m0t0aJcJ14y7ccsoGJsr+MrHOz5OQ7WWAF8q
FPcVEA/sWL+wRjkOK2dMtosCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAh8Cgv29v
Y+J0HRsrSNGLn5J8z/vU1vna0K14LvsyWLfa6W9Wif8kKNgTplSi3FJZnlmHHttL
gKQoxJAq6qQ5/Z2EgDUqUJf7LuAeVzCYbN3fbf3lFvYtOQv438j0ckPXsSwpQU+i
vI+ry3FTa4KqpCqzpsso9JkkzHhCqhIRJOS0inxPkQLVmHvxbFjMYCLRwEM/wfaL
QbfV9tpzXKbiJpkWdkMe6AyglNar/FU50lkUm7dTVdjvrW7ZJ6zN4l6tRpMrBJcc
2HS+lRCeifvPH8wq0XVnNUThTaJU91DzyrN1GqXQqduhe7FsL4dqMYT7Cqzm0neR
RAgDoFQpXWPZcQ==
-----END CERTIFICATE-----
subject=/C=AU/ST=Some-State/O=Splunk/CN=*.local
issuer=/C=AU/ST=Some-State/O=Splunk/CN=*.local
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2250 bytes and written 431 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: A71E3F0100E72CA2EBD8C4E2B38148A6BD8D1BDF7CB59E684C71A27B171BE54D
    Session-ID-ctx: 
    Master-Key: A96AA2D016D688AA631158C1E10D80CEE87C303C521E207D458A8484E3C04E4EB09216E1B4FB02FDD3A0C19AD81B43C4
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 41 4c a2 39 1e 22 44 4b-4c 97 c4 d5 49 19 f8 dc   AL.9."DKL...I...
    0010 - 95 07 72 3c 33 fd 80 ef-6e 67 4f d2 3d 99 a6 f4   ..r<3...ngO.=...
    0020 - 08 cd 06 aa 56 ed 6d 13-28 84 d1 e3 8d 45 1e 40   ....V.m.(....E.@
    0030 - ad 89 84 6d bf 80 ee 49-e0 4c 99 50 61 22 7b 23   ...m...I.L.Pa"{#
    0040 - 83 a4 03 7d 13 c7 1c 38-a7 86 82 fc b8 66 1e 22   ...}...8.....f."
    0050 - df a7 28 ae d1 b6 17 4e-fa 8c ba 46 44 e5 25 a3   ..(....N...FD.%.
    0060 - 21 b8 a8 5f b8 39 c0 7c-6f f8 5d 38 9a df 02 d9   !.._.9.|o.]8....
    0070 - 77 65 a9 32 d8 df 9b 84-3a 07 de 40 e4 6e 15 d6   we.2....:..@.n..
    0080 - 29 16 b4 b9 76 2c c2 5b-85 07 12 4c 32 18 2b d6   )...v,.[...L2.+.
    0090 - b0 d6 4e f6 ae e5 04 68-54 47 cc ce 9b 1c 71 7b   ..N....hTG....q{
    00a0 - a7 e4 73 01 44 2e b6 3d-52 fc 88 dc 45 c6 b7 5e   ..s.D..=R...E..^
    00b0 - bf 4e 10 97 ee ae 71 7c-33 41 54 62 46 20 85 86   .N....q|3ATbF ..

    Start Time: 1562352257
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---
read:errno=0

Re: Consume API of a remote hosted splunk

tbhasme — Fri, 05 Jul 2019 19:05:53 GMT

Now, I just opened the url https://splunk.local:8089/services/auth/login (which was showing the CN error) in another tab which showed me security risk. I accepted it and refreshed the page. Now there is no error but again, the response is empty and of type cors.

Re: Consume API of a remote hosted splunk

jkat54 — Fri, 05 Jul 2019 19:20:01 GMT

Did you try putting this certificate in your trusted root store?

Re: Consume API of a remote hosted splunk

jkat54 — Fri, 05 Jul 2019 19:20:55 GMT

And yes, when working with certificates like this, you'll want to completely close and open your browser between tests.

Re: Consume API of a remote hosted splunk

tbhasme — Sat, 06 Jul 2019 03:01:08 GMT

Yes, I installed the certificate in browser as well as system's trusted root store. I did restart browser and even incognito mode.

Re: Consume API of a remote hosted splunk

jkat54 — Sat, 06 Jul 2019 03:17:56 GMT

Check this out, you need to set the CORS policy in your request headers.

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Re: Consume API of a remote hosted splunk

jkat54 — Wed, 30 Sep 2020 01:11:20 GMT

I might be wrong there...

You did this already right? And restarted splunk? Make sure no typos please:

If you are using CORS, you'll need to configure the Splunk server to allow your web site to communicate with it using a CORS connection. To do this, add your web site's address as a trusted HTTP origin to the crossOriginSharingPolicy attribute in the server.conf configuration file.

For example, add this stanza to the $SPLUNK_HOME/etc/system/local/server.conf configuration file, then restart Splunk:

[httpServer]
crossOriginSharingPolicy = your_site_address

Re: Consume API of a remote hosted splunk

tbhasme — Mon, 08 Jul 2019 21:30:39 GMT

Yes, that was the second step of my setup... and yes, I did restart splunkd service...

Re: Consume API of a remote hosted splunk

jkat54 — Mon, 08 Jul 2019 22:22:54 GMT

Can you share the print out from this command?

 $splunk_home/bin/splunk btool server list httpServer  --debug

Re: Consume API of a remote hosted splunk

tbhasme — Mon, 08 Jul 2019 22:34:48 GMT

/opt/splunk/etc/system/local/server.conf   [httpServer]
/opt/splunk/etc/system/default/server.conf acceptFrom = *
/opt/splunk/etc/system/default/server.conf allowBasicAuth = true
/opt/splunk/etc/system/default/server.conf allowCookieAuth = true
/opt/splunk/etc/system/default/server.conf allowEmbedTokenAuth = true
/opt/splunk/etc/system/default/server.conf atomFeedStylesheet = /static/atom.xsl
/opt/splunk/etc/system/default/server.conf basicAuthRealm = /splunk
/opt/splunk/etc/system/default/server.conf busyKeepAliveIdleTimeout = 12
/opt/splunk/etc/system/default/server.conf cookieAuthHttpOnly = true
/opt/splunk/etc/system/default/server.conf cookieAuthSecure = true
/opt/splunk/etc/system/local/server.conf   crossOriginSharingPolicy = *
/opt/splunk/etc/system/default/server.conf dedicatedIoThreads = 0
/opt/splunk/etc/system/default/server.conf follow-symlinks = false
/opt/splunk/etc/system/default/server.conf forceHttp10 = auto
/opt/splunk/etc/system/default/server.conf keepAliveIdleTimeout = 7200
/opt/splunk/etc/system/default/server.conf max-age = 3600
/opt/splunk/etc/system/default/server.conf maxSockets = 0
/opt/splunk/etc/system/default/server.conf maxThreads = 0
/opt/splunk/etc/system/default/server.conf max_content_length = 2147483648
/opt/splunk/etc/system/default/server.conf streamInWriteTimeout = 5
/opt/splunk/etc/system/default/server.conf x_frame_options_sameorigin = true

Re: Consume API of a remote hosted splunk

jkat54 — Tue, 09 Jul 2019 00:11:50 GMT

Doesn't make much sense then does it? You should probably open a ticket.

Re: Consume API of a remote hosted splunk

jkat54 — Tue, 09 Jul 2019 00:32:40 GMT

What version of splunk are you using?

Afaik this will only work if you have splunk enterprise.

Re: Consume API of a remote hosted splunk

tbhasme — Fri, 12 Jul 2019 19:53:26 GMT

ok.. so turns out, changing the way to call the api from UI needed a different approach (not sure why)... right now only additional setting in the splunk server I have is

[sslConfig]
enableSplunkdSSL = false
.
.
.
[httpServer]
crossOriginSharingPolicy = *

and ui code is now using async method to fetch the token:

getToken = async() => {

        var details = {
          'username': 'admin',
          'password': 'J17kP3xjyBSvvKSuR5HrEYSI8w4'
      };

      var formBody = [];
      for (var property in details) {
        var encodedKey = encodeURIComponent(property);
        var encodedValue = encodeURIComponent(details[property]);
        formBody.push(encodedKey + "=" + encodedValue);
      }
      formBody = formBody.join("&");
      console.log(formBody)

        const response = await fetch("http://splunk.local:8089/services/auth/login?output_mode=json", {
          method: "POST",
          body: formBody,
          headers: { 
              'Content-type': 'application/x-www-form-urlencoded' 
            }
        });
        const responseInJson = await response.json();
        console.log(`Session Key - ${responseInJson.sessionKey}`); 
        return responseInJson.sessionKey
      }

And it somehow now seems to work!

The same server settings do not work with the earlier UI code but work with async somehow!