We are having a problem where requests are being sent to webservices but never return. I want to get a list of session numbers that sent a request to the webservice (XML Sent to Service) but did not get a response back (XML Response from Services). The following two lines are from Splunk that indexed our log4j logs of a successful purchase.
2013-07-26 14:48:45 - DEBUG:4LW5DNw1MdB5Rt5RhFUQCA__ - ExternalProcessManager.sendRequestToServices:(213) | XML Response from Services <?xml version="1.0" encoding="UTF-8"?>
<purchaseresponse haserror="0">
<price>39.02</price>
<product>G-200HA</product>
<actioncode>New</actioncode>
Show all 49 lines
host=CA1CW-WEB01 Options| sourcetype=log4j Options| source=ca1cw-web01\logs\log4j\csatp.2013-07-26.log Options
2013-07-26 14:48:45 - DEBUG:4LW5DNw1MdB5Rt5RhFUQCA__ - ExternalProcessManager.sendRequestToServices:(202) | XML Sent to Service http:/oursite.com/quote-rs/purchase: <?xml version="1.0" encoding="UTF-8"?>
<request><purchaserequest><actioncode>New</actioncode><aff>ha</aff><producer>HOMERTBO</producer><productclass>HOMEAWAYCP</productclass><plancode>A1</plancode>
Show all 49 lines
host=CA1CW-WEB01 Options| sourcetype=log4j Options| source=ca1cw-web01\logs\log4j\csatp.2013-07-26.log
How could we structure a query that uses rex to get the session number, then search for any session id's that did not log an (XML Response from Services)? I have tried this query and dont really know where to go from here:
sourcetype=log4j | rex ":\d\d - \S+:(?<sessionid>\S+) -.*<request><purchaserequest>" | search <response><purchaseresponse>
First, are you sure that your sessionid
is extracting properly from both the request and response events?
Next, this may be closer to what you want...
sourcetype=log4j "purchaserequest" OR "purchaseresponse"
| rex "\:\d\d - S+:(?<sessionid>S+) -.*<purchaserequest>"
| eval request=if(match(_raw,"\<purchaserequest\>",1,0)
| eval response=if(match(_raw,"\<purchaseresponse haserror=.*?\>",1,0)
| stats sum(request) as numRequests sum(response) as numReponses by sessionid
| where numRequests > numResponses
First, are you sure that your sessionid
is extracting properly from both the request and response events?
Next, this may be closer to what you want...
sourcetype=log4j "purchaserequest" OR "purchaseresponse"
| rex "\:\d\d - S+:(?<sessionid>S+) -.*<purchaserequest>"
| eval request=if(match(_raw,"\<purchaserequest\>",1,0)
| eval response=if(match(_raw,"\<purchaseresponse haserror=.*?\>",1,0)
| stats sum(request) as numRequests sum(response) as numReponses by sessionid
| where numRequests > numResponses
I am using this query to see by day what sessions had requests without responses:
sourcetype=log4j "XML Sent to Service" OR "XML Response from Services"
| search "purchaserequest" OR "purchaseresponse"
| rex "(?\d\d\d\d\-\d\d\-\d\d) .* -\s+\S+:(?[^ ]+) -.*"
| eval request=if(match(_raw,"\"),1,0)
| eval response=if(match(_raw,"\"),1,0)
| stats sum(request) as numRequests sum(response) as numResponses by date, sessionid
| where numRequests > numResponses
| sort date
Dang I love this product. There is always a way to do something.
You were right, I was missing a backslash in front of the S+ expression and that allowed logging of the session id. Also thanks for suggesting filtering out results before I process them, that speeds up the query quite a bit.
Then taking what you gave me and slightly tweaking it gave me some useful results.
I cannot update the original post, keep getting this field is required and all fields are filled in. The query should be this: sourcetype=log4j | rex "\:\d\d - \S+:(?