Splunk Search

Extract and cleanse data into field for table

nbhat
Explorer

Hi,

In the following log entries, I wanted to extract uri in a specific format:


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v2/clients?skip=0top=100,MediaType=null,XRemoteIP=null"


log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/clients/234,MediaType=null,XRemoteIP=null"

log: a_level="INFO", a_time="null", a_type="type", a_msg="Method=GET,Uri=http://monolith-xxx.abc.com/v1/users/123,MediaType=null,XRemoteIP=null"

For uri, I wanted the full extract until "?" or ",". Also remove and guids and digits from URL except for "/v1/","/v2/"
http://monolith-xxx.abc.com/v2/clients
http://monolith-xxx.abc.com/v1/clients/
http://monolith-xxx.abc.com/v1/users/

My current splunk query is as below:

index=aws_abc env=prd-01 uri Method StatusCode ResponseTimeMs
| rex field=log "ResponseTimeMs=(?<ResponseTimeMs>\d+),StatusCode=(?<StatusCode>\d+)"
| rex field=log "\"?Method\"?\=(?<Method>[^,]*)"
| rex field=log "Uri=(?<uri>[^\,]+)"
| rex field=uri mode=sed "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}|\d*//g"
| table uri,Method,StatusCode,ResponseTimeMs

I get value in the table for all 4 but uri in table shows as below
http://monolith-xxx.abc.com/v/clients?isactive=true

http://monolith-xxx.abc.com/v/users/?filter=(Name%startswith%'H')


Expected Output:
http://monolith-xxx.abc.com/v2/clients

http://monolith-xxx.abc.com/v2/users/

Please help. Thanks

Labels (3)
0 Karma

diogofgm
SplunkTrust
SplunkTrust

The sed your using its to broad and its picking up de version number aswell

Try something like this:

| rex "Uri=(?<uri>.*)[\/?]"

If you want to keep the last / you can try something like this:

|rex "Uri=(?<uri>http:\/(?:\/[^\/\?]+){3}\/?)"

 

------------
Hope I was able to help you. If so, some karma would be appreciated.
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...