Splunk Search

How to rex?

mikeyty07
Communicator

I have 2 kind of logs where there are two types of uri which i want to rex into different fields

{logType=DOWNSTREAM_RESPONSE, requestUri=https://google.come.com:8000/google/api/updateapi?&lo=en_US&sc=RT, duration=22, requestId=znXdSxbJQw6iVTtEeykZVA, globalTrackingId=null, requestTrackingId=null, request={body={"a":{"b":{"country":"US", }}}, method=POST, requestUri=https://google.come.com:443/google/api/updateapi?&lo=en_US&sc=RT}, response=(200 OK, { "body="{} }, "headers="{}, "statusCode=OK", statusCodeValue=200}")"}

{logType=DOWNSTREAM_RESPONSE, requestUri=https://google.come.com:8000/google/api/deleteapi, duration=33, requestId=asdasd, globalTrackingId=null, requestTrackingId=null, request={body={"a":{"b":{"country":"US", }}}, method=POST, requestUri=https://google.come.com:443/google/api/updateapi?&lo=en_US&sc=RT}, response=(200 OK, { "body="{} }, "headers="{}, "statusCode=OK", statusCodeValue=200}")"}

http= https

URL= google.come.com:8000

service = /google

api= /api/updateapi
api= /api/deleteapi

params= ?&lo=en_US&sc=RT

is there a way to regex this?

 

Labels (4)
1 Solution

Tom_Lundie
Contributor

Hi,

Yes you can extract those fields using | rex.

Try this:

| rex max_match=0 "(?<http>https?)\:\/\/(?<URL>[^\/]*)(?<service>/[^\/]*)(?<api>[^\?\,\}]*)(?:\?(?<params>[^\,\}]*))?"

This will extract all URLs in the _raw event into those fields that you suggested. If you want to run this on a specific field then you could add the field=<<field>> argument to the | rex command.

View solution in original post

inventsekar
Ultra Champion

Hi @mikeyty07 ... its pretty simple actually.. check this out.. 

 

source="two-logs.txt" host="laptop" sourcetype="twologs" 
| rex field=_raw "requestUri\=(?P<status>\w+)\:\/\/(?P<URL>\w+\.\w+\.\w+\:\d+)(?P<service>\/\w+)(?P<api>\/\w+\/\w+)[\,|\?](?P<params>\S+)\,"  
| table status URL service api params

 

check the sample run:

rex-two-logs.jpg

 

Tom_Lundie
Contributor

Hi,

Yes you can extract those fields using | rex.

Try this:

| rex max_match=0 "(?<http>https?)\:\/\/(?<URL>[^\/]*)(?<service>/[^\/]*)(?<api>[^\?\,\}]*)(?:\?(?<params>[^\,\}]*))?"

This will extract all URLs in the _raw event into those fields that you suggested. If you want to run this on a specific field then you could add the field=<<field>> argument to the | rex command.

mikeyty07
Communicator

if i have to add the duration in same query of rex what would it be?

0 Karma

Tom_Lundie
Contributor

Looking at your sample logs, duration is a separate field and it only seems to be applied in context of one of your URLs so I wouldn't extract it within the same | rex query.

Best-practise here is to set-up search-time field extractions for these fields. Read more here. The general idea is to make your fields extract automatically so that you don't have to run these rex commands for every single-use case that pertains to these logs.

That being said, if you're sure that you want to get this extracted via rex, use a separate command, the following regex will work:

| rex "duration=(?<duration>\d+)"

 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...