I have below HTTP events where in I am trying to extract status code, response time and URL. I am using the following rex query below. This query works fine to find error 200, 400 and 500. But not for 30* errors. If you refer the below 302 events its has event like (302 - ). other status has like(200 87909) (400 568). Can you help me with the expression that is missing so that it extract for all the codes. I verfied it in "https://regex101.com/r/bVp3gz/1" as well.
===============
HTTP\/1.1\"\s(?\d+)\s(?\d+)\s"(?[^\"]*)"
==========================
11.111.111.1 [04/Mar/2021:09:05:40 -0600] 1061614 "GET /merced/content/frag/breeze/bootstrap/fonts/icomoon.ttf?az1hj2 HTTP/1.1" 200 95364 "https://sfdfdsfsd-sfsdfasf.topms.com/mxxx/treports/prepackaged/O-Rx_Agent_MyUnacknowledgedCoachingSe..." "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" - B6613D2EAB90BFAB32BD90BA61E8280E.app1
11.111.111.11 [04/Mar/2021:09:36:41 -0600] 169017 "GET /delegate/forwarderServlet/process.do?url=%2Fmerced%2Fdashboards%2FO-Rx_Agent_HomePage_Dash%3Flang%3Den_US&appid=xxx HTTP/1.1" 302 - "https://sfdfdsfsd-sfsdfasf.topms.com/group/npm/o-rx_agent_homepage_dash" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.190 Safari/537.36" - C9A3D61145B15DEFCD8BD71736242EA8.tomcat2
11.111.111.1 [04/Mar/2021:08:35:20 -0600] 17580 "GET /merced/populate?assistant=person&query=jomalyn%2520mallari&policyName=%2Fcom%2Fmerced%2Fmodels%2Femployee%2Fpolicies%2FCoachingWritePolicy&fieldName=EEDRFE HTTP/1.1" 500 977 "https://sfdfdsfsd-sfsdfasf.topms.com/mxxx/forms/BPLCoachingSessionForm?lang=en_US" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" - ABEAA3DED8CBC4163D544C986AA455DA.app4
11.111.111.1 [04/Mar/2021:10:00:27 -0600] 0 "GET /nice-documentation/javascripts/MercedHelpLib.js?browserId=other&minifierType=js&languageId=en_US&b=0000&t=1612576281967 HTTP/1.1" 404 1083 "https://sfdfdsfsd-sfsdfasf.topms.com/group/xxxx/o-rx_agent_homepage_dash" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" - A8B66CD6680C8747D4C878CAAE64B1D7.tomcat1
Try this:
| rex "\<Health\>\s\<(?<host>[^\>]+)\>\s\<(?<container>[^\>]+)\>.*\<(?<memory>\d+)\%\sof\sthe\stotal\smemory"
If this reply helps you, an upvote/like would be appreciated.
Sorry about missing escape and quotes;
| rex "HTTP\/1.1\"\s(?<status>\d+)\s(?<restime>\d+|-)\s\"(?<url>[^\"]*)"
Hi @ravir_jbp,
Please try using below, response time is not decimal on 302 messages,
rex HTTP\/1.1\"\s(?<status>\d+)\s(?<restime>\d+|-)\s"(?<url>[^\"]*)"
I am getting this error:
Error in 'SearchParser': Missing a search command before '-'. Error at position '80' of search query 'search index=cba_XXXXX_XXXX |rex HTTP\/1.1\"\s(...{snipped} {errorcontext = stime>\d+|-)\s"(?<url}'.
Looks like you are missing a backslash before the double quote at the start of the URL
Try this:
| rex "HTTP\/1.1\"\s(?<status_code>\d+)\s(?<response_time>(?:\d+|\-))\s\"(?<URL>[^\"]*)\""
Hi,
Sorry I posted the wrong query that I wrote. The one I am using is:
Rex HTTP\/1.1\"\s(?<status>\d+)\s(?<restime>\d+)\s"(?<url>[^\"]*)"
This one is not working for below 302 event. For other its owrking.
11.111.111.11 [04/Mar/2021:09:36:41 -0600] 169017 "GET /delegate/forwarderServlet/process.do?url=%2Fmerced%2Fdashboards%2FO-Rx_Agent_HomePage_Dash%3Flang%3Den_US&appid=xxx HTTP/1.1" 302 - "https://sfdfdsfsd-sfsdfasf.topms.com/group/npm/o-rx_agent_homepage_dash" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.190 Safari/537.36" - C9A3D61145B15DEFCD8BD71736242EA8.tomcat2
Did you try the regex I posted? It works for logs containing 302 - also.
Hello Manjunath,
Yes it worked for my search. I was able to get the diffeent status codes.
index=XX-XX_XX |rex "HTTP\/1.1\"\s(?<status_code>\d+)\s(?<response_time>(?:\d+|\-))\s\"(?<URL>[^\"]*)\"" | stats count by status_code
Thank you Manjunath.
Need your help on the requirement. Curently I am using the rex query you have provided to search for different status successfully. Now I am trying to get the top 10 URL which were accessed by user but got error messages. May be with 500 or 400 or 300 status code URLs.
index=XXXXXXXX |rex "HTTP\/1.1\"\s(?<status_code>\d+)\s(?<response_time>(?:\d+|\-))\s\"(?<URL>[^\"]*)\"" |
Use top command:
| top URL
Hello manjunath, THat work worked. THank you for quick response. I have another question related to Rex as mentioned below
event:
=========
<Mar 8, 2021 03:16:12,638 CST> <Info> <Health> <hostname> <containername> <weblogic.GCMonitor> <<anonymous>> <> <e1d36038-3bd0-4ace-beb3-b9b78707786e-00000006> <1615194972638> <[severity-value: 64] [rid: 0:1] [partition-id: 0] [partition-name: DOMAIN] > <BEA-310002> <88% of the total memory in the server is free.>
I am also trying to use rex for following event but the previous one is not working.
I am trying to find the memory %for each "<containername>" for each "<host>". for above event, 88% is total memory for specific <containername> and <hostname>. Please help how to exact the memory % from above event.
Hello @manjunathmeti ,
Actually I like to extract the memory for each container for every host. One host may have multiple containers. In below example events I have four event (for two hosts) but has two containers in one host (contianer name is agent-desktop3 and avgent-desktop2). I am trying to exact in below table format:
Host name | container | memory
<Mar 9, 2021 02:30:29,585 CST> <Info> <Health> <hostname1.host.com> <agent-desktop3> <weblogic.GCMonitor> <<anonymous>> <> <d19b10c2-4d5d-43ad-a4f5-bc0dc842a71b-00000007> <1615278629585> <[severity-value: 64] [rid: 0:1] [partition-id: 0] [partition-name: DOMAIN] > <BEA-310002> <44% of the total memory in the server is free.>
####<Mar 9, 2021 02:30:11,563 CST> <Info> <Health> <hostname1.host.com> <agent-desktop2> <weblogic.GCMonitor> <<anonymous>> <> <e1d36038-3bd0-4ace-beb3-b9b78707786e-00000006> <1615278611563> <[severity-value: 64] [rid: 0:1] [partition-id: 0] [partition-name: DOMAIN] > <BEA-310002> <58% of the total memory in the server is free.>
<Mar 9, 2021 02:30:29,585 CST> <Info> <Health> <hostname2.host.com> <agent-desktop1> <weblogic.GCMonitor> <<anonymous>> <> <d19b10c2-4d5d-43ad-a4f5-bc0dc842a71b-00000007> <1615278629585> <[severity-value: 64] [rid: 0:1] [partition-id: 0] [partition-name: DOMAIN] > <BEA-310002> <44% of the total memory in the server is free.>
####<Mar 9, 2021 02:30:11,563 CST> <Info> <Health> <hostname2.host.com> <agent-desktop2> <weblogic.GCMonitor> <<anonymous>> <> <e1d36038-3bd0-4ace-beb3-b9b78707786e-00000006> <1615278611563> <[severity-value: 64] [rid: 0:1] [partition-id: 0] [partition-name: DOMAIN] > <BEA-310002> <58% of the total memory in the server is free.>
Try this:
| rex "\<Health\>\s\<(?<host>[^\>]+)\>\s\<(?<container>[^\>]+)\>.*\<(?<memory>\d+)\%\sof\sthe\stotal\smemory"
If this reply helps you, an upvote/like would be appreciated.
You need a different regex for this. Try this:
| rex "\<(?<memory>\d+)\%\sof\sthe\stotal\smemory"
If these replies help you, upvote/likes would be appreciated.