topic Re: how to rex in Splunk Search

How to rex?

mikeyty07 — Thu, 23 Mar 2023 14:35:55 GMT

I have 2 kind of logs where there are two types of uri which i want to rex into different fields

{logType=DOWNSTREAM_RESPONSE, requestUri=https://google.come.com:8000/google/api/updateapi?&lo=en_US&sc=RT, duration=22, requestId=znXdSxbJQw6iVTtEeykZVA, globalTrackingId=null, requestTrackingId=null, request={body={"a":{"b":{"country":"US", }}}, method=POST, requestUri=https://google.come.com:443/google/api/updateapi?&lo=en_US&sc=RT}, response=(200 OK, { "body="{} }, "headers="{}, "statusCode=OK", statusCodeValue=200}")"}

{logType=DOWNSTREAM_RESPONSE, requestUri=https://google.come.com:8000/google/api/deleteapi, duration=33, requestId=asdasd, globalTrackingId=null, requestTrackingId=null, request={body={"a":{"b":{"country":"US", }}}, method=POST, requestUri=https://google.come.com:443/google/api/updateapi?&lo=en_US&sc=RT}, response=(200 OK, { "body="{} }, "headers="{}, "statusCode=OK", statusCodeValue=200}")"}

http= https

URL= google.come.com:8000

service = /google

api= /api/updateapi
api= /api/deleteapi

params= ?&lo=en_US&sc=RT

is there a way to regex this?

Re: how to rex

Tom_Lundie — Wed, 22 Mar 2023 23:26:06 GMT

Hi,

Yes you can extract those fields using | rex.

Try this:

| rex max_match=0 "(?<http>https?)\:\/\/(?<URL>[^\/]*)(?<service>/[^\/]*)(?<api>[^\?\,\}]*)(?:\?(?<params>[^\,\}]*))?"

This will extract all URLs in the _raw event into those fields that you suggested. If you want to run this on a specific field then you could add the field=<<field>> argument to the | rex command.

Re: how to rex

inventsekar — Thu, 23 Mar 2023 00:07:25 GMT

Hi @mikeyty07 ... its pretty simple actually.. check this out..

source="two-logs.txt" host="laptop" sourcetype="twologs" | rex field=_raw "requestUri\=(?P<status>\w+)\:\/\/(?P<URL>\w+\.\w+\.\w+\:\d+)(?P<service>\/\w+)(?P<api>\/\w+\/\w+)[\,|\?](?P<params>\S+)\," | table status URL service api params

check the sample run:

Re: how to rex

mikeyty07 — Thu, 23 Mar 2023 15:24:15 GMT

if i have to add the duration in same query of rex what would it be?

Re: how to rex

Tom_Lundie — Thu, 23 Mar 2023 16:14:22 GMT

Looking at your sample logs, duration is a separate field and it only seems to be applied in context of one of your URLs so I wouldn't extract it within the same | rex query.

Best-practise here is to set-up search-time field extractions for these fields. Read more here. The general idea is to make your fields extract automatically so that you don't have to run these rex commands for every single-use case that pertains to these logs.

That being said, if you're sure that you want to get this extracted via rex, use a separate command, the following regex will work:

| rex "duration=(?<duration>\d+)"