topic Re: Can splunk Index data from an HTTP address in Getting Data In

Can splunk Index data from an HTTP address

htkhtk — Wed, 29 Sep 2010 22:45:37 GMT

I have a url that has a server.txt file on that that gets updated as the requests/responses are coming through.

Can splunk index that http://website.com/server.txt address?

If so, how? Thanks!

Re: Can splunk Index data from an HTTP address

ziegfried — Wed, 29 Sep 2010 22:58:41 GMT

Sure, just set up a scripted input, that fetches the url.

Dokumentation: http://www.splunk.com/base/Documentation/4.1.5/Admin/Setupcustom(scripted)inputs

As an example, you could use the curl command on Linux/Unix:

#!/bin/bash
curl http://website.com/server.txt

Re: Can splunk Index data from an HTTP address

gkanapathy — Thu, 30 Sep 2010 11:10:48 GMT

There is also http://www.splunkbase.com/apps/All/app:Web+Page+Monitor

Re: Can splunk Index data from an HTTP address

sfurness — Fri, 13 May 2011 20:55:52 GMT

Is Web Page Monitor still available? For some reason I can't find it in the Apps section?

Re: Can splunk Index data from an HTTP address

gkanapathy — Fri, 13 May 2011 20:57:28 GMT

Yes, I was looking for it, and it used to do just what was requested, when it was around.

Re: Can splunk Index data from an HTTP address

nicholasgrabows — Thu, 27 Dec 2012 20:00:03 GMT

There is a new splunkbase app called "importutil". It lets you import input from an http url via the splunk search command line. Also works for ftp. sftp is experimental.

http://splunk-base.splunk.com/apps/69078/importutil

Here is an ftp example. Pulling from the bureau of labor stats:

|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes

Here is an http example that imports data from the federal reserve economic data website:

|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE

Re: Can splunk Index data from an HTTP address

pabstec — Wed, 06 Feb 2013 18:47:05 GMT

Check out the "rest" command.