HTTPS authenticated input source

[vijayansundarar]

I have some data that I can access from Web Browser (via authenticated HTTPS). The data is plain text. I would like to use this data as an input source form my Splunk server.

Is there an app that can do this? Is there a way to accomplish this?

[MuS]

Hi vijayansundararajan,

have you tried the App Web Page Monitor? Within it you can choose whether it will index the page content or not. It supports basic authentication as well as HTTPS.

Otherwise you can write a Python script which will grab the Webpage and do what ever you want with it within Splunk.

hope this helps...

cheers, MuS

[MuS]

take a look at the script used by this App, it is a simple Python script using the urllib2 module. For Docs about urllib2 module goto http://docs.python.org/2/library/urllib2.html

[vijayansundarar]

Is there any documentation/instructions for the basic authentication?

[Damien_Dallimor · Accepted solution]

Although it is not REST per say , it is most likely HTTP(s) Get , you should be able to leverage my REST API Modular Input to poll this data for you.

This modular input also supports all standard authentication schemes.

[Damien_Dallimor]

Modular Inputs were not available in the product until version 5.

[vijayansundarar]

Have you had any luck with this on Splunk 4.3.2?

[lukejadamec]

If you cannot access the server that holds the data and install/configure a forwarder on it with ssl certificates, then your best bet would be to go to the site, down load the data from the site with the secure https connection and save it to a computer that does have a forwarder.

Verify the data is indexable (csv, key=value, etc...) and then monitor that directory/file.

[vijayansundarar]

I should have mentioned, that I this is actually a log file and it is continuous. Will this still work? Do I need to provide the difference or can Splunk handle this like it does a file that is forwarded?

[ddarmand]

you can monitore a file in Manager > Data inputs

[vijayansundarar]

Do you mean this will work via https with authentication? I do not have access to the filesystem on the remote machine.