topic Re: Get list of session id's that did not get a response from web service. in Splunk Search

Get list of session id's that did not get a response from web service.

phoeniix — Fri, 26 Jul 2013 23:06:25 GMT

We are having a problem where requests are being sent to webservices but never return. I want to get a list of session numbers that sent a request to the webservice (XML Sent to Service) but did not get a response back (XML Response from Services). The following two lines are from Splunk that indexed our log4j logs of a successful purchase.

2013-07-26 14:48:45 - DEBUG:4LW5DNw1MdB5Rt5RhFUQCA__ - ExternalProcessManager.sendRequestToServices:(213) | XML Response from Services <?xml version="1.0" encoding="UTF-8"?>
<purchaseresponse haserror="0">
<price>39.02</price>
<product>G-200HA</product>
<actioncode>New</actioncode>

Show all 49 lines

host=CA1CW-WEB01 Options| sourcetype=log4j Options| source=ca1cw-web01\logs\log4j\csatp.2013-07-26.log Options

2013-07-26 14:48:45 - DEBUG:4LW5DNw1MdB5Rt5RhFUQCA__ - ExternalProcessManager.sendRequestToServices:(202) | XML Sent to Service http:/oursite.com/quote-rs/purchase: <?xml version="1.0" encoding="UTF-8"?>
<request><purchaserequest><actioncode>New</actioncode><aff>ha</aff><producer>HOMERTBO</producer><productclass>HOMEAWAYCP</productclass><plancode>A1</plancode>

Show all 49 lines

host=CA1CW-WEB01 Options| sourcetype=log4j Options| source=ca1cw-web01\logs\log4j\csatp.2013-07-26.log

How could we structure a query that uses rex to get the session number, then search for any session id's that did not log an (XML Response from Services)? I have tried this query and dont really know where to go from here:

sourcetype=log4j | rex ":\d\d - \S+:(?<sessionid>\S+) -.*<request><purchaserequest>" | search <response><purchaseresponse>

Re: Get list of session id's that did not get a response from web service.

phoeniix — Fri, 26 Jul 2013 23:22:54 GMT

I cannot update the original post, keep getting this field is required and all fields are filled in. The query should be this: sourcetype=log4j | rex "\:\d\d - \S+:(?\S+) -.*<purchaserequest>" | search <purchaseresponse>

Re: Get list of session id's that did not get a response from web service.

lguinn2 — Sat, 27 Jul 2013 01:17:00 GMT

First, are you sure that your sessionid is extracting properly from both the request and response events?

Next, this may be closer to what you want...

sourcetype=log4j "purchaserequest" OR "purchaseresponse"
| rex "\:\d\d - S+:(?<sessionid>S+) -.*<purchaserequest>" 
| eval request=if(match(_raw,"\<purchaserequest\>",1,0)
| eval response=if(match(_raw,"\<purchaseresponse haserror=.*?\>",1,0)
| stats sum(request) as numRequests sum(response) as numReponses by sessionid
| where numRequests > numResponses

Re: Get list of session id's that did not get a response from web service.

phoeniix — Mon, 29 Jul 2013 23:22:01 GMT

Dang I love this product. There is always a way to do something.

You were right, I was missing a backslash in front of the S+ expression and that allowed logging of the session id. Also thanks for suggesting filtering out results before I process them, that speeds up the query quite a bit.

Then taking what you gave me and slightly tweaking it gave me some useful results.

Re: Get list of session id's that did not get a response from web service.

phoeniix — Mon, 29 Jul 2013 23:22:15 GMT

I am using this query to see by day what sessions had requests without responses:

sourcetype=log4j "XML Sent to Service" OR "XML Response from Services" 
| search "purchaserequest" OR "purchaseresponse"
| rex "(?\d\d\d\d\-\d\d\-\d\d) .* -\s+\S+:(?[^ ]+) -.*" 
| eval request=if(match(_raw,"\"),1,0)
| eval response=if(match(_raw,"\"),1,0)
| stats sum(request) as numRequests sum(response) as numResponses by date, sessionid
| where numRequests > numResponses
| sort date