Splunk Search

Extract and cleanse data into field for table

nbhat
Explorer

Hi,

In the following log entries, I wanted to extract uri in a specific format:


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v2/clients?skip=0top=100,MediaType=null,XRemoteIP=null"


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/clients/234,MediaType=null,XRemoteIP=null"

log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/users/123,MediaType=null,XRemoteIP=null"

For uri, I wanted the full extract until "?" or ",". Also remove and guids and digits from URL except for "/v1/","/v2/"
http://monolith-xxx.abc.com/v2/clients
http://monolith-xxx.abc.com/v1/clients/
http://monolith-xxx.abc.com/v1/users/

My current splunk query is as below:

index=aws_abc env=prd-01 uri Method StatusCode ResponseTimeMs
| rex field=log "ResponseTimeMs=(?<ResponseTimeMs>\d+),StatusCode=(?<StatusCode>\d+)"
| rex field=log "\"?Method\"?\=(?<Method>[^,]*)"
| rex field=log "Uri=(?<uri>[^\,]+)"
| rex field=uri mode=sed "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}|\d*//g"
| table uri,Method,StatusCode,ResponseTimeMs

I get value in the table for all 4 but uri in table shows as below
http://monolith-xxx.abc.com/v/clients?isactive=true

http://monolith-xxx.abc.com/v/users/?filter=(Name%startswith%'H')


Expected Output:
http://monolith-xxx.abc.com/v2/clients

http://monolith-xxx.abc.com/v2/users/

Please help. Thanks

Labels (3)
0 Karma

diogofgm
SplunkTrust
SplunkTrust

The sed your using its to broad and its picking up de version number aswell

Try something like this:

| rex "Uri=(?<uri>.*)[\/?]"

If you want to keep the last / you can try something like this:

|rex "Uri=(?<uri>http:\/(?:\/[^\/\?]+){3}\/?)"

 

------------
Hope I was able to help you. If so, some karma would be appreciated.
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...