topic Re: Call-home failure when using Cisco Networks App. in All Apps and Add-ons

Call-home failure when using Cisco Networks App.

mileven — Thu, 12 Mar 2015 16:56:58 GMT

I have configured the Cisco App and have the Cisco TA installed. When I go and run call-home send alert-group inventory Profile Splunk I get an error within Splunk:

HTTP_REQUEST_FAILED: failed to send  HTTP request to :

on the local syslog I get the following:

Mar 12 2015 16:49:20.693 UTC: %CALL_HOME-3-HTTP_REQUEST_FAILED: failed to send HTTP request to :
    http://10.196.173.41:516 
        (ERR 112 : Connection time out)
Mar 12 2015 16:49:20.693 UTC: %CALL_HOME-3-ONDEMAND_MESSAGE_FAILED: call-home on-demand message failed to send (ERR 112, Connection time out)

On the switch which is a Catalyst 6509E, I can telnet to the heavy forwarder and the tcp port I defined for this feature.

I have it configured as defined in the help file. My source interface is my default management vlan.

Re: Call-home failure when using Cisco Networks App.

mikaelbje — Thu, 12 Mar 2015 21:47:54 GMT

The call home feature is a little bit flaky and doesn't work well on the lower end devices such as the 2960 series (no periodic call home), but since you have a 6500 series it should work, unless there's a software defect.

So you configured the http source interface as something like the following, right?

ip http client source-interface Loopback10

By the look of the error message it seems that there is no connectivity between the Splunk server and the devicw. When you tried to open a telnet session from the device did you specify the same source interface and/or VRF you need?

If the source interface in question belongs to a VRF maybe the ip http client source-interface doesn't honor the VRF (potential IOS software bug)

You could also try to increase the response timeout for HTTP sessions:

ip http client response timeout 180

Any luck?

Re: Call-home failure when using Cisco Networks App.

nmethod — Mon, 28 Sep 2020 20:09:52 GMT

Was his ever solved? I am having the same problem, with the same configuration as you seem to have. It looks to me like the issue has something to do with CallHome waiting for a response, and not receiving it.

Relevant debug from a 3560x:
014071: CALL-HOME-TRACE: RESPONSE DATA STARTS HERE
014072: CALL-HOME-TRACE: call_home_http_resp_data() status = 6
014073: CALL-HOME-TRACE: call_home_set_httpc_resp_status: is entered. tid (9), status (111), err_string ()
014074: CALL-HOME-DETAIL: call_home_set_httpc_resp_status: set tid (9) httpc resp stat (111) err string (Response time out)
014075: CALL-HOME-TRACE: call_home_http_resp_data : set tid (9) resp status to (111)
014076: CALL-HOME-TRACE: call_home_wait_for_httpc_resp : unblocked wait for tid (9), status is (111)err string is (Response time out)
014077: CALL-HOME-TRACE: call_home_remove_httpc_resp_node : is entered
014078: CALL-HOME-TRACE: call_home_httpc_resp_stat_clean_up is entered
014079: CALL-HOME-TRACE: http resp from http://10.10.90.101:1337 failed, tid (9), response status (111), err string (Response time out)
014087: Jun 2 11:12:52.098 EDT: %CALL_HOME-3-HTTP_REQUEST_FAILED: failed to send HTTP request to :
http://10.10.90.101:1337 (ERR 111 : Response time out)

Re: Call-home failure when using Cisco Networks App.

mikaelbje — Tue, 02 Jun 2015 16:10:22 GMT

Have you tried doing a packet capture with tcpdump/wireshark on the server where your TCP port is defined? Do you see anything?

The Splunk server won't really give any kind of response to the client, except a TCP ACK when the session starts. The port doesn't honor any HTTP specs. It's not a web server, it just receives streamed data, in this case XML. It should work unless Cisco changed their Call Home specification to require some kind of application acknowledgement. Again this functionality is not perfect in all IOS versions. Try IOS 15.x if you're not already on it.

Re: Call-home failure when using Cisco Networks App.

tomasmoser — Tue, 29 Sep 2020 12:46:34 GMT

Hi,

Have you solved the problem? I followed instructions for an add-on installation and it seems I am getting getting Reset-I from Splunk after IOS tries to send SCH XML batch of information. IOS SCH Error code 110 is not documented anywhere 😞

IOS router
Syslog output:
Feb 9 15:31:06.116: %CALL_HOME-3-HTTP_REQUEST_FAILED: failed to send HTTP request to :
http://1.1.1.1:8989
(ERR 110 : Request Aborted)

IOS configuration:
call-home
contact-email-addr someone@somewhere.eu
street-address "address bla bla"
site-id "OUR-SITE-ID"
source-ip-address "10.10.10.10"
profile "Splunk"
destination transport-method http
destination address http http://1.1.1.1:8989
subscribe-to-alert-group configuration
subscribe-to-alert-group environment severity debug
subscribe-to-alert-group inventory
subscribe-to-alert-group syslog severity debug pattern ".*"
subscribe-to-alert-group inventory periodic daily 15:32

Splunk Heavy Forwarder (receiver of SHC messages)
inputs.conf:
[tcp://1.1.1.1:8989]
host = host1
connection_host = none
sourcetype = Cisco:SmartCallHome

[tcp://1.1.1.2:8989]
host = host2
connection_host = none
sourcetype = Cisco:SmartCallHome

Is really configuration on Splunk enough? Just tcp stanza and nothing else?

Tomas

Re: Call-home failure when using Cisco Networks App.

mikaelbje — Fri, 10 Feb 2017 12:18:43 GMT

Make sure you have the TA-cisco_ios add-on installed on your Heavy Forwarder to correctly handle line breaks. Maybe that helps?

Re: Call-home failure when using Cisco Networks App.

mikaelbje — Fri, 10 Feb 2017 12:19:35 GMT

And you can always test from your Cisco device by doing: "telnet 1.1.1.1 8089" if the connection really works. Anything you write on this socket will end up in Splunk.

Re: Call-home failure when using Cisco Networks App.

tomasmoser — Fri, 10 Feb 2017 12:23:39 GMT

Yes, I did that later during troubleshooting too.

Re: Call-home failure when using Cisco Networks App.

tomasmoser — Fri, 10 Feb 2017 12:37:23 GMT

I can confirm that SCH is set up correctly. Testing works. We do not know why automatic SCH sending does not work.

Successfully verified with:
call-home test "test2" profile Splunk
telnet 1.1.1.1 8989
...sending bla bla

Thanks!

Tomas