topic Re: Can you index images with Splunk? in Getting Data In

Can you index images with Splunk?

chustar — Fri, 21 Aug 2015 18:55:58 GMT

Is it possible to index images in splunk?
I want to gather logs from a certain location, so I specified an index like this:

[monitor://\\path\to\monitor]
whitelist=LOGFILE

However, that location will contain multiple types of data, specifically, logs and images. I want to use those images in my splunk dashboards, so I changed my index.conf to look like this:

[monitor://\\path\to\monitor]
whitelist=(LOGFILE|screenshot\.png)

This doesn't seem to work. I looked into using fschange but it looks like you can't use monitor and fschange on the same directory (according to: http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf?utm_source=answers&utm_medium=incontext&utm_term=inputs.conf&utm_campaign=refdoc)

Re: Can you index images with Splunk?

somesoni2 — Fri, 21 Aug 2015 19:24:31 GMT

I don't think so Splunk is a tool for this requirement (indexing image). I believe you'd need some bigdata solution to store that and then you can use Splunk HUNK to do image search.

Re: Can you index images with Splunk?

jensonthottian — Fri, 21 Aug 2015 19:50:13 GMT

A creative solution :

If you want to monitor the images using Splunk.
Create an active script which will update a text file whenever an image is added into a directory prefixing it with the timestamp(add size if you want) the image was put into the directory.
Then index this text file in Splunk 🙂

Re: Can you index images with Splunk?

chustar — Fri, 21 Aug 2015 21:14:32 GMT

I really just wanted the file names so I can embed them in the dashboard. No need for bigdata solutions

Re: Can you index images with Splunk?

woodcock — Fri, 21 Aug 2015 22:53:27 GMT

Are you sure Splunk is the right tool for this situation? Whenever people are working with documents, I usually suggest MarkLogic which has tools to help you generate the metadata that you are describing. It is an incredible product and does things in a totally different way than Splunk and is better suited for non-plain-text data sources:
http://www.marklogic.com

P.S. These are the main guys that swooped in and made HealthCare.gov actually work; without them, it probably never would have.

Re: Can you index images with Splunk?

Damien_Dallimor — Fri, 21 Aug 2015 23:16:52 GMT

You could use the Command Modular Input to execute a command that looks in a directory , finds images files and indexes those image file paths/names in Splunk.

image_lister.sh

#!/bin/sh

find /Users/ddallimore/Desktop -type f -exec file {} \; | awk -F: '{ if ($2 ~/[Ii]mage|EPS/) print $1}'

Setup a Command Modular Input stanza to fire the image_lister.sh command

Search in Splunk

Re: Can you index images with Splunk?

ChrisG — Sat, 22 Aug 2015 00:21:09 GMT

Fair point, but I think you mean MarkLogic, not Mark/Space. http://www.marklogic.com/customers/healthcare-gov/

Re: Can you index images with Splunk?

woodcock — Sat, 22 Aug 2015 14:17:44 GMT

Thank you for the correction; I have updated my answer.