httpstatus command: Why does the command return "0" when any path is added to to the URL?

[jensonthottian]

This command works great with url as "www.splunk.com" - return status code as "200",
But return http status as "0" when any path is added - i.e.“www.splunk.com/en_us/products.html”

index=abc| eval url="www.splunk.com" |httpstatus |table url httpstatus

Below is the python script :

import sys,splunk.Intersplunk
import string
import httplib

urlfield="url"

if len(sys.argv)>1 and len(sys.argv) != 4:
    print "Usage |httpstatus url as <local-field> (or have url field name in data)"
    sys.exit()
elif len(sys.argv) == 4:
    urlfield=sys.argv[3]

results = []

try:

results,dummyresults,settings = splunk.Intersplunk.getOrganizedResults()

for r in results:
    if "_raw" in r:
        if urlfield in r:
            try:
                conn = httplib.HTTPConnection(r[urlfield], timeout=5)
                conn.request("HEAD","")
                res = conn.getresponse()
                r["httpstatus"] = res.status
                conn.close()
            except:
                r["httpstatus"] = "0"
                if (conn != None):
                    conn.close()
except:
    import traceback
    stack =  traceback.format_exc()
    results = splunk.Intersplunk.generateErrorResults("Error : Traceback: " + str(stack))

splunk.Intersplunk.outputResults( results )

[richgalloway · Accepted solution]

The httpConnection() method expects either hostname or hostname:port. It does not work with a path. Pass the full URL (with path) as the second argument to the request() method.

[jensonthottian]

The addition of path as second argument to the request method--

Broke the URL field in two parts - host and path
And created another variable host

conn = httplib.HTTPConnection(r[urlfield], timeout=5)
conn.request("HEAD", path)
return conn.getresponse().status
except StandardError:
return None