Splunk Search

Get list of session id's that did not get a response from web service.

Engager

We are having a problem where requests are being sent to webservices but never return. I want to get a list of session numbers that sent a request to the webservice (XML Sent to Service) but did not get a response back (XML Response from Services). The following two lines are from Splunk that indexed our log4j logs of a successful purchase.

2013-07-26 14:48:45 - DEBUG:4LW5DNw1MdB5Rt5RhFUQCA__ - ExternalProcessManager.sendRequestToServices:(213) | XML Response from Services <?xml version="1.0" encoding="UTF-8"?>
<purchaseresponse haserror="0">
<price>39.02</price>
<product>G-200HA</product>
<actioncode>New</actioncode>

Show all 49 lines

host=CA1CW-WEB01 Options| sourcetype=log4j Options| source=ca1cw-web01\logs\log4j\csatp.2013-07-26.log Options

2013-07-26 14:48:45 - DEBUG:4LW5DNw1MdB5Rt5RhFUQCA__ - ExternalProcessManager.sendRequestToServices:(202) | XML Sent to Service http:/oursite.com/quote-rs/purchase: <?xml version="1.0" encoding="UTF-8"?>
<request><purchaserequest><actioncode>New</actioncode><aff>ha</aff><producer>HOMERTBO</producer><productclass>HOMEAWAYCP</productclass><plancode>A1</plancode>

Show all 49 lines

host=CA1CW-WEB01 Options| sourcetype=log4j Options| source=ca1cw-web01\logs\log4j\csatp.2013-07-26.log

How could we structure a query that uses rex to get the session number, then search for any session id's that did not log an (XML Response from Services)? I have tried this query and dont really know where to go from here:

sourcetype=log4j | rex ":\d\d - \S+:(?<sessionid>\S+) -.*&lt;request&gt;&lt;purchaserequest&gt;" | search &lt;response&gt;&lt;purchaseresponse&gt;

Tags (3)
0 Karma
1 Solution

Legend

First, are you sure that your sessionid is extracting properly from both the request and response events?

Next, this may be closer to what you want...

sourcetype=log4j "purchaserequest" OR "purchaseresponse"
| rex "\:\d\d - S+:(?<sessionid>S+) -.*<purchaserequest>" 
| eval request=if(match(_raw,"\<purchaserequest\>",1,0)
| eval response=if(match(_raw,"\<purchaseresponse haserror=.*?\>",1,0)
| stats sum(request) as numRequests sum(response) as numReponses by sessionid
| where numRequests > numResponses

View solution in original post

Legend

First, are you sure that your sessionid is extracting properly from both the request and response events?

Next, this may be closer to what you want...

sourcetype=log4j "purchaserequest" OR "purchaseresponse"
| rex "\:\d\d - S+:(?<sessionid>S+) -.*<purchaserequest>" 
| eval request=if(match(_raw,"\<purchaserequest\>",1,0)
| eval response=if(match(_raw,"\<purchaseresponse haserror=.*?\>",1,0)
| stats sum(request) as numRequests sum(response) as numReponses by sessionid
| where numRequests > numResponses

View solution in original post

Engager

I am using this query to see by day what sessions had requests without responses:

sourcetype=log4j "XML Sent to Service" OR "XML Response from Services" 
| search "purchaserequest" OR "purchaseresponse"
| rex "(?\d\d\d\d\-\d\d\-\d\d) .* -\s+\S+:(?[^ ]+) -.*" 
| eval request=if(match(_raw,"\"),1,0)
| eval response=if(match(_raw,"\"),1,0)
| stats sum(request) as numRequests sum(response) as numResponses by date, sessionid
| where numRequests > numResponses
| sort date
0 Karma

Engager

Dang I love this product. There is always a way to do something.

You were right, I was missing a backslash in front of the S+ expression and that allowed logging of the session id. Also thanks for suggesting filtering out results before I process them, that speeds up the query quite a bit.

Then taking what you gave me and slightly tweaking it gave me some useful results.

0 Karma

Engager

I cannot update the original post, keep getting this field is required and all fields are filled in. The query should be this: sourcetype=log4j | rex "\:\d\d - \S+:(?\S+) -.*<purchaserequest>" | search <purchaseresponse>

0 Karma