Splunk Search

Extract and cleanse data into field for table

nbhat
Explorer

Hi,

In the following log entries, I wanted to extract uri in a specific format:


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v2/clients?skip=0top=100,MediaType=null,XRemoteIP=null"


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/clients/234,MediaType=null,XRemoteIP=null"

log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/users/123,MediaType=null,XRemoteIP=null"

For uri, I wanted the full extract until "?" or ",". Also remove and guids and digits from URL except for "/v1/","/v2/"
http://monolith-xxx.abc.com/v2/clients
http://monolith-xxx.abc.com/v1/clients/
http://monolith-xxx.abc.com/v1/users/

My current splunk query is as below:

index=aws_abc env=prd-01 uri Method StatusCode ResponseTimeMs
| rex field=log "ResponseTimeMs=(?<ResponseTimeMs>\d+),StatusCode=(?<StatusCode>\d+)"
| rex field=log "\"?Method\"?\=(?<Method>[^,]*)"
| rex field=log "Uri=(?<uri>[^\,]+)"
| rex field=uri mode=sed "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}|\d*//g"
| table uri,Method,StatusCode,ResponseTimeMs

I get value in the table for all 4 but uri in table shows as below
http://monolith-xxx.abc.com/v/clients?isactive=true

http://monolith-xxx.abc.com/v/users/?filter=(Name%startswith%'H')


Expected Output:
http://monolith-xxx.abc.com/v2/clients

http://monolith-xxx.abc.com/v2/users/

Please help. Thanks

Labels (3)
0 Karma

diogofgm
SplunkTrust
SplunkTrust

The sed your using its to broad and its picking up de version number aswell

Try something like this:

| rex "Uri=(?<uri>.*)[\/?]"

If you want to keep the last / you can try something like this:

|rex "Uri=(?<uri>http:\/(?:\/[^\/\?]+){3}\/?)"

 

------------
Hope I was able to help you. If so, some karma would be appreciated.
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>