Splunk Search

Extract and cleanse data into field for table

nbhat
Explorer

Hi,

In the following log entries, I wanted to extract uri in a specific format:


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v2/clients?skip=0top=100,MediaType=null,XRemoteIP=null"


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/clients/234,MediaType=null,XRemoteIP=null"

log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/users/123,MediaType=null,XRemoteIP=null"

For uri, I wanted the full extract until "?" or ",". Also remove and guids and digits from URL except for "/v1/","/v2/"
http://monolith-xxx.abc.com/v2/clients
http://monolith-xxx.abc.com/v1/clients/
http://monolith-xxx.abc.com/v1/users/

My current splunk query is as below:

index=aws_abc env=prd-01 uri Method StatusCode ResponseTimeMs
| rex field=log "ResponseTimeMs=(?<ResponseTimeMs>\d+),StatusCode=(?<StatusCode>\d+)"
| rex field=log "\"?Method\"?\=(?<Method>[^,]*)"
| rex field=log "Uri=(?<uri>[^\,]+)"
| rex field=uri mode=sed "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}|\d*//g"
| table uri,Method,StatusCode,ResponseTimeMs

I get value in the table for all 4 but uri in table shows as below
http://monolith-xxx.abc.com/v/clients?isactive=true

http://monolith-xxx.abc.com/v/users/?filter=(Name%startswith%'H')


Expected Output:
http://monolith-xxx.abc.com/v2/clients

http://monolith-xxx.abc.com/v2/users/

Please help. Thanks

Labels (3)
0 Karma

diogofgm
SplunkTrust
SplunkTrust

The sed your using its to broad and its picking up de version number aswell

Try something like this:

| rex "Uri=(?<uri>.*)[\/?]"

If you want to keep the last / you can try something like this:

|rex "Uri=(?<uri>http:\/(?:\/[^\/\?]+){3}\/?)"

 

------------
Hope I was able to help you. If so, some karma would be appreciated.
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...