Crawling URL and analizing contents by Splunk

[vccsupport]

Hello,
I look for Apps which performs the crawl of Web Site of the Link point automatically from Top Page of the URL.
I would like to use this Apps with Splunk and to analyze contents from Web Site of its crawl result.
If any solution, please inform. It would be appreciated.

[MuS]

Hi vccsupport

you can use the webmon App for this, you will be able to crawl web pages with it and index the result. If you don't like it, you could write your own python script using urllib2 to read the webpage and use it as scripted input - that what I was doing lately 😉

cheers,
MuS

[vccsupport]

Thanks for useful information.
I expect for n level crawling.

VCC support