Getting Data In

Return binary data (such as images) via REST API or otherwise

Builder

I'm trying to make some custom extensions to our application, with some additional html divs displaying images. The application will get those images from an external image server and generate some URL to serve as src= in img HTML tag. As a possibility, I can envision our python script (invoked via REST API?) storing images somewhere under appserver/static and generating the proper URLs to access them.

Are there any ready examples demonstrating something like that? I can try to use pdfgen_endpoint.py as an example, but that might take a while...

Tags (3)
1 Solution

Builder

OK, this exercise is done. Here are my steps:
- Create an application named test_rest_api.
- Create the following stanza in $SPLUNK_HOME/etc/system/local/web.conf:

 [expose:test_rest_api]
 methods = GET
 pattern = test_rest_api/test
  • Create the following stanza in $SPLUNK_HOME/etc/system/local/restmap.conf:

    [script:testRestApi]
    match = /test_rest_api/test
    scripttype = python
    handler = test_rest_script.MyImageHandler
    requireAuthentication = false

Create a python script named test_rest_script.py in etc/apps/test_rest_api/bin folder). Here is the entire script:

import splunk
import splunk.rest as rest
import splunk.entity as entity
import splunk.auth
import splunk.models.dashboard as sm_dashboard
import splunk.models.dashboard_panel as sm_dashboard_panel
import splunk.models.saved_search as sm_saved_search
import splunk.search
import splunk.search.searchUtils
from splunk.util import normalizeBoolean

import os, sys

class MyImageHandler(splunk.rest.BaseRestHandler):
    TEST_API_BINARY_MODE = 'binary'
    BINARY_FILE_NAME = '../etc/apps/test_rest_api/appserver/static/rubber-duck.jpg' # relative to $SPLUNK_HOME/bin
    message = 'Test <strong>REST</strong> API'
    status = 200

    def handle_GET(self):
        if self.TEST_API_BINARY_MODE in self.args and self.args.get(self.TEST_API_BINARY_MODE) in  ('true', 'True', 'y', 'yes', '1'):
            self._binResponse()
        else:
            self._textResponse()

    def _textResponse(self):
        self.response.write(self.message)
        self.response.setStatus(self.status)
        self.response.setHeader("content-type", "text/html")

    def _binResponse(self):
        try:
            data = self._getImage()
            self.response.write(data)
            self.response.setStatus(self.status)
            self.response.setHeader('content-type','image/jpg')
        except Exception as e:
            self.message = str(e)
            self.status = 501
            self._textResponse()

    def _getFileName(self):
        scriptname = os.path.abspath(sys.argv[0])
        scriptdir = os.path.dirname(scriptname)
        imgname = os.path.normpath('/'.join([ scriptdir, self.BINARY_FILE_NAME ]))
        return imgname

    def _getImage(self):
        imgname = self._getFileName()
        f = open(imgname, 'rb')
        data = f.read()
        f.close()
        return data

Why is sys.argv[0] returning a $SPLUNK_HOME/bin folder rather than etc/apps/test_rest_api/bin, I don't know, but it's used for testing purposes only (I've put that rubber-duck.jpg in my app's appserver/static). My real script will read the image from an http link :).

So you just enter https://localhost:8089/services/test_rest_api/test?binary=true and enjoy the picture of the rubber duck! Or omit that binary parameter (or, say, put 'false' there) and look at the proud 'Test REST API' message.

View solution in original post

Builder

OK, this exercise is done. Here are my steps:
- Create an application named test_rest_api.
- Create the following stanza in $SPLUNK_HOME/etc/system/local/web.conf:

 [expose:test_rest_api]
 methods = GET
 pattern = test_rest_api/test
  • Create the following stanza in $SPLUNK_HOME/etc/system/local/restmap.conf:

    [script:testRestApi]
    match = /test_rest_api/test
    scripttype = python
    handler = test_rest_script.MyImageHandler
    requireAuthentication = false

Create a python script named test_rest_script.py in etc/apps/test_rest_api/bin folder). Here is the entire script:

import splunk
import splunk.rest as rest
import splunk.entity as entity
import splunk.auth
import splunk.models.dashboard as sm_dashboard
import splunk.models.dashboard_panel as sm_dashboard_panel
import splunk.models.saved_search as sm_saved_search
import splunk.search
import splunk.search.searchUtils
from splunk.util import normalizeBoolean

import os, sys

class MyImageHandler(splunk.rest.BaseRestHandler):
    TEST_API_BINARY_MODE = 'binary'
    BINARY_FILE_NAME = '../etc/apps/test_rest_api/appserver/static/rubber-duck.jpg' # relative to $SPLUNK_HOME/bin
    message = 'Test <strong>REST</strong> API'
    status = 200

    def handle_GET(self):
        if self.TEST_API_BINARY_MODE in self.args and self.args.get(self.TEST_API_BINARY_MODE) in  ('true', 'True', 'y', 'yes', '1'):
            self._binResponse()
        else:
            self._textResponse()

    def _textResponse(self):
        self.response.write(self.message)
        self.response.setStatus(self.status)
        self.response.setHeader("content-type", "text/html")

    def _binResponse(self):
        try:
            data = self._getImage()
            self.response.write(data)
            self.response.setStatus(self.status)
            self.response.setHeader('content-type','image/jpg')
        except Exception as e:
            self.message = str(e)
            self.status = 501
            self._textResponse()

    def _getFileName(self):
        scriptname = os.path.abspath(sys.argv[0])
        scriptdir = os.path.dirname(scriptname)
        imgname = os.path.normpath('/'.join([ scriptdir, self.BINARY_FILE_NAME ]))
        return imgname

    def _getImage(self):
        imgname = self._getFileName()
        f = open(imgname, 'rb')
        data = f.read()
        f.close()
        return data

Why is sys.argv[0] returning a $SPLUNK_HOME/bin folder rather than etc/apps/test_rest_api/bin, I don't know, but it's used for testing purposes only (I've put that rubber-duck.jpg in my app's appserver/static). My real script will read the image from an http link :).

So you just enter https://localhost:8089/services/test_rest_api/test?binary=true and enjoy the picture of the rubber duck! Or omit that binary parameter (or, say, put 'false' there) and look at the proud 'Test REST API' message.

View solution in original post

Builder

I found out why sys.argv[0] has $SPLUNK_HOME/bin folder in the script name. The script being run is in fact $SPLUNK_HOME/bin/runSript.py, with the parameter of test_rest_script.MyImageHandler (script name.class name). Makes sense...

0 Karma

Builder

Status update:

So far, I learned how to create an endpoint and tie it to a script. I did the following:

  1. Created an application with no views (Visible: false) named test_rest_api (very creative name, I know).
  2. Added an [expose:...] stanza to web.conf (if you don't have one in etc/system/local, create it - just don't modify the one in etc/system/default!). The stanza looked like this:

    [expose:test_rest_api]
    methods = GET
    pattern = test_rest_api/test

  3. Added a [script:...] stanza to the restmap.conf in etc/system/local. It looked like this:

    [script:testRestApi]
    match = /test_rest_api/test
    scripttype = python
    handler = test_rest_script.MyImageHandler
    requireAuthentication = false

  4. Created a script named test_rest_script.py (very creative again) in etc/apps/test_rest_api/bin. The file looks like this so far:

    import splunk
    import splunk.rest as rest
    import splunk.entity as entity
    import splunk.auth
    import splunk.models.dashboard as sm_dashboard
    import splunk.models.dashboard_panel as sm_dashboard_panel
    import splunk.models.saved_search as sm_saved_search
    import splunk.search
    import splunk.search.searchUtils
    from splunk.util import normalizeBoolean

    class MyImageHandler(splunk.rest.BaseRestHandler):
    TEST_API_BINARY_MODE = 'binary'

    def handle_GET(self):
        self._textResponse()
    
    def _textResponse(self):
        self.response.write("Test <strong>REST</strong> API")
        self.response.setStatus(200)
        self.response.setHeader("content-type", "text/html")
    

I'm not sure I'll need all those imports - I mimicked pdfgen_endpoint.py in this. Anyway, my next step would be to put an image file somewhere within my app (probably in appserver/static, though not necessary), read that if the requested mode is binary and write it into the response. Will update once I succeed. Right now I do get that Test REST API on the screen when I enter https://localhost:8089/services/test_rest_api/test/ - with or without the trailing slash in the URL.

0 Karma

Builder

Another update - I got it all working. All that I still had to do was this:

        f = open(imgname, 'rb')
        data = f.read()
        f.close()
        self.response.write(data)
        self.response.setStatus(self.status)
        self.response.setHeader('content-type','image/jpg')

Here imgname was the file name of the image I wanted to show. I used content-type of image/jpg rather than application/octet-stream only because I tested that in Chrome and it does not show such content as image, instead offering to download it.
Anyway, this concludes this exercise. I guess in the end I answered my own question - again 🙂

0 Karma

Champion

Glad you are figuring things out. Here some points. When your finish I recommend posting answer to the question with you final result.

Champion

Are you asking how to create your own splunk rest endpoint?

0 Karma

Builder

In fact, both - how to create a custom endpoint and how to handle that in python. However, I did look deeper into pdfgen since I've sent this question, so now I have some crude idea. So far I see the following:
1) web.conf: [expose:uniqueName] stanza with pattern=... and methods=... at least
2) restmap.conf: [script:description] stanza with match=..., handler=... and requiresAuthentication=...
3) The Python script: a class matching the class name in handler in restmap.conf, with handle_GET etc. methods. The method returning a binary file would read it into a byte array, write that into the response, set the HTTP header content-type to application/octet-stream or similar and return.

Any pitfalls on this way?

0 Karma

Champion

Your heading down the right path. Here is another good example of an app using the restmap. https://splunkbase.splunk.com/app/1607/. You should join the IRC chat, lots of the experts hang out there.