All Apps and Add-ons

REST API Issue - Exception performing request

satishsdange
Builder

I am trying to pull data from Azure using REST API. Have installed REST API App https://apps.splunk.com/app/1546/ on the top of Splunk Enterprise.

After giving necessary credentials, I am getting following error in Splunkd :-

12-24-2014 04:11:32.173 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:

12-24-2014 04:11:22.157 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:

12-24-2014 04:11:12.141 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:

12-24-2014 04:11:02.125 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:

Could someone please advise me what could be the problem?

Thanks in advance.

0 Karma

Damien_Dallimor
Ultra Champion

As per my request , can you provide a link "detailing the actual specifications for the Azure REST endpoint you are trying to connect to"

https://msdn.microsoft.com/en-us/library/dd179355.aspx is an overview page. I have absolutely no idea what REST endpoint you are trying to hit.

Guessing , is it one of these specs ? https://msdn.microsoft.com/en-us/library/azure/ee460790.aspx

Based on the endpoint URL you have entered above in inputs.conf , maybe this ? https://msdn.microsoft.com/en-us/library/azure/ee460802.aspx

If so , then you need to authenticate by sending a management certificate in the REST requests : https://msdn.microsoft.com/en-us/library/azure/ee460782.aspx#bk_cert

That link details how to create one. You'll then need to create a custom authentication handler in the REST Mod Input to read your certificate from the file system and add it to the REST request.

An example of a custom auth handler that you can add to rest_ta/bin/authhandlers.py is :

class MyAzureCertAuthHAndler(AuthBase):
    def __init__(self,**args):
        self.cert = args['certPath']
        pass

    def __call__(self, r):
        r.cert = self.cert
        return r

Then your setup might look like below. I don't know why you were using the http proxy setting , I've stripped that out.I also added HTTP Headers because it said so in the documentation I read.

alt text

0 Karma

satishsdange
Builder

I added management certification into Azure console & copied that on my Splunk instance. Did below changes in authhandlers.py

class newazurecert(AuthBase):
def init(self,**args):
# setup any auth-related data here
self.cert = args['certPath']
#self.password = args['password']
pass

Now I am getting 403 error.

1-22-2015 19:45:34.488 +0530 ERROR ExecProcessor - message from "python /Users/sdange/Documents/Splunk_Demo/Script_Splunk6.2/splunk/etc/apps/rest_ta/bin/rest.py" HTTP Request error: 403 Client Error: Forbidden

1/22/15

7:45:22.196 PM

01-22-2015 19:45:22.196 +0530 ERROR ExecProcessor - message from "python /Users/sdange/Documents/Splunk_Demo/Script_Splunk6.2/splunk/etc/apps/rest_ta/bin/rest.py" HTTP Request error: 403 Client Error: Forbidden

http://azure.microsoft.com/en-in/documentation/articles/storage-monitoring-diagnosing-troubleshootin... says that I have to reissue storage token key from which I am trying to pull data but I have not given these keys in REST API.

what could be the problem?

Thanks

0 Karma

Damien_Dallimor
Ultra Champion

1) were my assumptions in my answer correct ? you didn't concur

2) your auth handler looks nothing like the example I provided you. Furthermore without also seeing the inputs.conf stanza , I can't be of much use.

0 Karma

satishsdange
Builder

My mistake. Apologize for that. You are right. I referred the same link to enter endpoint URL.

Here is my authhandler.py configuration -

class newazurecert(AuthBase):
def init(self,**args):
# setup any auth-related data here
self.cert = args['certPath']
#self.password = args['password']
pass

inputs.conf

[rest://Azure]
auth_type = custom
custom_auth_handler = newazurecert
custom_auth_handler_args = certPath=/Users/sdange/Documents/Apps & TA/Azure/newazurecert.cer
endpoint = https://management.core.windows.net/XXXXX/services/storageservices/satishsplcs1
http_header_propertys = x-ms-version=2014-10-01,Content-Type=application/xml
http_method = GET
index = azure
index_error_response_codes = 0
response_type = text
sourcetype = csv
streaming_request = 0

0 Karma

Damien_Dallimor
Ultra Champion

Copy/Paste from my previous reply :

your auth handler looks nothing like the example I provided you

Repasting :

class MyAzureCertAuthHAndler(AuthBase):
     def __init__(self,**args):
         self.cert = args['certPath']
         pass

     def __call__(self, r):
         r.cert = self.cert
         return r

Furthermore , I don't know why you are specifying "sourcetype = csv". If you actually read the Azure docs you'll see that the response is XML.

0 Karma

satishsdange
Builder

Here is authhandler.py configuration -

class newazurecert(AuthBase):
def init(self,**args):
# setup any auth-related data here
self.cert = args['certPath']
#self.password = args['password']
pass

def __call__(self, r):
    r.cert = self.cert
    return r

inputs.conf

[rest://Azure]
auth_type = custom
custom_auth_handler = newazurecert
custom_auth_handler_args = certPath=/Users/sdange/Documents/Apps & TA/Azure/newazurecert.cer
endpoint = https://management.core.windows.net/XXXX/services/storageservices/satishsplcs1
http_header_propertys = x-ms-version=2014-10-01,Content-Type=application/xml
http_method = GET
index = azure
index_error_response_codes = 0
response_type = text
sourcetype = xml
streaming_request = 0

0 Karma

Damien_Dallimor
Ultra Champion

I'd remove the spaces in the cert path too.

0 Karma

satishsdange
Builder

sorry, I did not understand Damien.

0 Karma

Damien_Dallimor
Ultra Champion

certPath=/Users/sdange/Documents/Apps & TA/Azure/newazurecert.cer

You have spaces in the cert path. I'd remove these.

Beyond that , everything else in the config looks fine.

If you are still getting HTTP 403 responses , I'd refer to the Azure documentation at that point as to why your authentication for the request could be getting declined.

0 Karma

satishsdange
Builder

well, that too did not work.

Now I am referring https://msdn.microsoft.com/en-us/library/hh225339.aspx to get some pointers.

0 Karma

satishsdange
Builder

Can someone please advise me how to fix this problem?

0 Karma

sdange_splunk
Splunk Employee
Splunk Employee

any inputs on this question?

0 Karma

Damien_Dallimor
Ultra Champion

You are going to have to provide more concise information for us to help you debug.

1) Splunk version
2) REST Mod Input version
3) your inputs.conf stanza you setup (sensitive info masked out) . When you enter data via the REST Mod Input set page , it gets persisted to inputs.conf. Search under etc/apps for it.
4) any more descriptive errors , search in Splunk Web : "index=_internal ExecProcessor error rest.py"

0 Karma

Damien_Dallimor
Ultra Champion

Furthermore , in addition to "I am trying to pull data from Azure using REST API" , can you please provide a link to the Azure documentation detailing the actual specifications for the Azure REST endpoint you are trying to connect to.

0 Karma

satishsdange
Builder

Hi Damien -

Here is information

1) Splunk Version - 6.2.
2) REST Mod input version - 1.3.5 https://apps.splunk.com/app/1546/
3) your inputs.conf stanza you setup (sensitive info masked out) -
[rest://Azure]
auth_type = oauth2
endpoint = https://management.core.windows.net/XXXXX/services/storageservices/satishsplcs1
host =
http_method = GET
index = azure
index_error_response_codes = 0
oauth1_access_token = XXXXXXX
response_type = text
sourcetype = csv
streaming_request = 0
oauth1_access_token_secret = XXXXX
http_proxy = https://manage.windowsazure.com/@XXXXXX.onmicrosoft.com#Workspaces/All/dashboard
oauth2_access_token = XXXXXXXX
oauth2_client_secret = XXXXXXX
disabled = 0

4) any more descriptive errors , search in Splunk Web : "index=_internal ExecProcessor error rest.py”
01-21-2015 15:46:03.545 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:
host = satishsplindx source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
1/21/15
3:45:53.533 PM
01-21-2015 15:45:53.533 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/rest_ta/bin/rest.py" Exception performing request:
host = satishsplindx source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd

5) Azure link - https://msdn.microsoft.com/en-us/library/dd179355.aspx

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...