topic HTTPS authenticated input source in Getting Data In

HTTPS authenticated input source

vijayansundarar — Thu, 01 Aug 2013 11:10:07 GMT

I have some data that I can access from Web Browser (via authenticated HTTPS). The data is plain text. I would like to use this data as an input source form my Splunk server.

Is there an app that can do this? Is there a way to accomplish this?

Re: HTTPS authenticated input source

ddarmand — Thu, 01 Aug 2013 11:28:59 GMT

you can monitore a file in Manager > Data inputs

Re: HTTPS authenticated input source

lukejadamec — Thu, 01 Aug 2013 15:14:40 GMT

If you cannot access the server that holds the data and install/configure a forwarder on it with ssl certificates, then your best bet would be to go to the site, down load the data from the site with the secure https connection and save it to a computer that does have a forwarder.

Verify the data is indexable (csv, key=value, etc...) and then monitor that directory/file.

Re: HTTPS authenticated input source

vijayansundarar — Fri, 02 Aug 2013 02:06:10 GMT

Do you mean this will work via https with authentication? I do not have access to the filesystem on the remote machine.

Re: HTTPS authenticated input source

vijayansundarar — Fri, 02 Aug 2013 02:12:07 GMT

I should have mentioned, that I this is actually a log file and it is continuous. Will this still work? Do I need to provide the difference or can Splunk handle this like it does a file that is forwarded?

Re: HTTPS authenticated input source

Damien_Dallimor — Fri, 02 Aug 2013 03:16:53 GMT

Although it is not REST per say , it is most likely HTTP(s) Get , you should be able to leverage my REST API Modular Input to poll this data for you.

This modular input also supports all standard authentication schemes.

Re: HTTPS authenticated input source

vijayansundarar — Tue, 08 Oct 2013 09:43:36 GMT

Have you had any luck with this on Splunk 4.3.2?

Re: HTTPS authenticated input source

MuS — Tue, 08 Oct 2013 09:53:01 GMT

Hi vijayansundararajan,

have you tried the App Web Page Monitor? Within it you can choose whether it will index the page content or not. It supports basic authentication as well as HTTPS.

Otherwise you can write a Python script which will grab the Webpage and do what ever you want with it within Splunk.

hope this helps...

cheers, MuS

Re: HTTPS authenticated input source

vijayansundarar — Tue, 08 Oct 2013 10:39:20 GMT

Is there any documentation/instructions for the basic authentication?

Re: HTTPS authenticated input source

MuS — Tue, 08 Oct 2013 11:20:28 GMT

take a look at the script used by this App, it is a simple Python script using the urllib2 module. For Docs about urllib2 module goto http://docs.python.org/2/library/urllib2.html

Re: HTTPS authenticated input source

Damien_Dallimor — Tue, 08 Oct 2013 16:12:04 GMT

Modular Inputs were not available in the product until version 5.