Return binary data (such as images) via REST API or otherwise

[arkadyz1]

I'm trying to make some custom extensions to our application, with some additional html divs displaying images. The application will get those images from an external image server and generate some URL to serve as src= in img HTML tag. As a possibility, I can envision our python script (invoked via REST API?) storing images somewhere under appserver/static and generating the proper URLs to access them.

Are there any ready examples demonstrating something like that? I can try to use pdfgen_endpoint.py as an example, but that might take a while...

[arkadyz1 · Accepted solution]

OK, this exercise is done. Here are my steps:
- Create an application named test_rest_api.
- Create the following stanza in $SPLUNK_HOME/etc/system/local/web.conf:

 [expose:test_rest_api]
 methods = GET
 pattern = test_rest_api/test

• Create the following stanza in $SPLUNK_HOME/etc/system/local/restmap.conf:

  [script:testRestApi]
  match = /test_rest_api/test
  scripttype = python
  handler = test_rest_script.MyImageHandler
  requireAuthentication = false

Create a python script named test_rest_script.py in etc/apps/test_rest_api/bin folder). Here is the entire script:

import splunk
import splunk.rest as rest
import splunk.entity as entity
import splunk.auth
import splunk.models.dashboard as sm_dashboard
import splunk.models.dashboard_panel as sm_dashboard_panel
import splunk.models.saved_search as sm_saved_search
import splunk.search
import splunk.search.searchUtils
from splunk.util import normalizeBoolean

import os, sys

class MyImageHandler(splunk.rest.BaseRestHandler):
    TEST_API_BINARY_MODE = 'binary'
    BINARY_FILE_NAME = '../etc/apps/test_rest_api/appserver/static/rubber-duck.jpg' # relative to $SPLUNK_HOME/bin
    message = 'Test <strong>REST</strong> API'
    status = 200

    def handle_GET(self):
        if self.TEST_API_BINARY_MODE in self.args and self.args.get(self.TEST_API_BINARY_MODE) in  ('true', 'True', 'y', 'yes', '1'):
            self._binResponse()
        else:
            self._textResponse()

    def _textResponse(self):
        self.response.write(self.message)
        self.response.setStatus(self.status)
        self.response.setHeader("content-type", "text/html")

    def _binResponse(self):
        try:
            data = self._getImage()
            self.response.write(data)
            self.response.setStatus(self.status)
            self.response.setHeader('content-type','image/jpg')
        except Exception as e:
            self.message = str(e)
            self.status = 501
            self._textResponse()

    def _getFileName(self):
        scriptname = os.path.abspath(sys.argv[0])
        scriptdir = os.path.dirname(scriptname)
        imgname = os.path.normpath('/'.join([ scriptdir, self.BINARY_FILE_NAME ]))
        return imgname

    def _getImage(self):
        imgname = self._getFileName()
        f = open(imgname, 'rb')
        data = f.read()
        f.close()
        return data

Why is sys.argv[0] returning a $SPLUNK_HOME/bin folder rather than etc/apps/test_rest_api/bin, I don't know, but it's used for testing purposes only (I've put that rubber-duck.jpg in my app's appserver/static). My real script will read the image from an http link :).

So you just enter https://localhost:8089/services/test_rest_api/test?binary=true and enjoy the picture of the rubber duck! Or omit that binary parameter (or, say, put 'false' there) and look at the proud 'Test REST API' message.

[arkadyz1]

I found out why sys.argv[0] has $SPLUNK_HOME/bin folder in the script name. The script being run is in fact $SPLUNK_HOME/bin/runSript.py, with the parameter of test_rest_script.MyImageHandler (script name.class name). Makes sense...

[arkadyz1]

Status update:

So far, I learned how to create an endpoint and tie it to a script. I did the following:

1. Created an application with no views (Visible: false) named test_rest_api (very creative name, I know).

2. Added an [expose:...] stanza to web.conf (if you don't have one in etc/system/local, create it - just don't modify the one in etc/system/default!). The stanza looked like this:

   [expose:test_rest_api]
   methods = GET
   pattern = test_rest_api/test

3. Added a [script:...] stanza to the restmap.conf in etc/system/local. It looked like this:

   [script:testRestApi]
   match = /test_rest_api/test
   scripttype = python
   handler = test_rest_script.MyImageHandler
   requireAuthentication = false

4. Created a script named test_rest_script.py (very creative again) in etc/apps/test_rest_api/bin. The file looks like this so far:

   import splunk
   import splunk.rest as rest
   import splunk.entity as entity
   import splunk.auth
   import splunk.models.dashboard as sm_dashboard
   import splunk.models.dashboard_panel as sm_dashboard_panel
   import splunk.models.saved_search as sm_saved_search
   import splunk.search
   import splunk.search.searchUtils
   from splunk.util import normalizeBoolean

   class MyImageHandler(splunk.rest.BaseRestHandler):
   TEST_API_BINARY_MODE = 'binary'

   def handle_GET(self):
       self._textResponse()
   
   def _textResponse(self):
       self.response.write("Test <strong>REST</strong> API")
       self.response.setStatus(200)
       self.response.setHeader("content-type", "text/html")
   

I'm not sure I'll need all those imports - I mimicked pdfgen_endpoint.py in this. Anyway, my next step would be to put an image file somewhere within my app (probably in appserver/static, though not necessary), read that if the requested mode is binary and write it into the response. Will update once I succeed. Right now I do get that Test REST API on the screen when I enter https://localhost:8089/services/test_rest_api/test/ - with or without the trailing slash in the URL.

[arkadyz1]

Another update - I got it all working. All that I still had to do was this:

        f = open(imgname, 'rb')
        data = f.read()
        f.close()
        self.response.write(data)
        self.response.setStatus(self.status)
        self.response.setHeader('content-type','image/jpg')

Here imgname was the file name of the image I wanted to show. I used content-type of image/jpg rather than application/octet-stream only because I tested that in Chrome and it does not show such content as image, instead offering to download it.
Anyway, this concludes this exercise. I guess in the end I answered my own question - again 🙂

[bmacias84]

Glad you are figuring things out. Here some points. When your finish I recommend posting answer to the question with you final result.

[bmacias84]

Are you asking how to create your own splunk rest endpoint?

[arkadyz1]

In fact, both - how to create a custom endpoint and how to handle that in python. However, I did look deeper into pdfgen since I've sent this question, so now I have some crude idea. So far I see the following:
1) web.conf: [expose:uniqueName] stanza with pattern=... and methods=... at least
2) restmap.conf: [script:description] stanza with match=..., handler=... and requiresAuthentication=...
3) The Python script: a class matching the class name in handler in restmap.conf, with handle_GET etc. methods. The method returning a binary file would read it into a byte array, write that into the response, set the HTTP header content-type to application/octet-stream or similar and return.

Any pitfalls on this way?

[bmacias84]

Your heading down the right path. Here is another good example of an app using the restmap. https://splunkbase.splunk.com/app/1607/. You should join the IRC chat, lots of the experts hang out there.