Splunk Search

How to create new field based on regex?

Loves-to-Learn Everything

I have a log stream in this format:

level=info request.elapsed=100 request.method=GET request.path=/orders/123456 request_id=2ca011b5-ad34-4f32-a95c-78e8b5b1a270 response.status=500

I have extracted the fields using regex:

| rex field=message "level=info request.elapsed=(?<duration>.*) request.method=(?<method>.*) request.path=(?<path>.*) request_id=(?<request_id>.*) response.status=(?<statusCode>.*)"

I want to manually build a new field called route based on the extracted field path. For example, for "path=/order/123456", I want to create new field "route=/order/{orderID}", so I can grouping by route not by path, the path contains real parameter which I cannot group on path. 

How can I achieve this? Thanks.

Labels (2)
0 Karma


I do not see why you needed to do that extra extraction because Splunk should have given you a field named "request_path" already. (See emulation below.)  All you need to do is to assign a new field based on match.


| eval route = if(match(request_path, "^/orders/\d+"), "/order/{orderID}", null())


The sample data should give you something like


Is this what you wanted?

Here is a data emulation you can play with and compare with real data.


| makeresults
| eval _raw = "level=info request.elapsed=100 request.method=GET request.path=/orders/123456 request_id=2ca011b5-ad34-4f32-a95c-78e8b5b1a270 response.status=500"
| extract
``` data emulation above ```


Of course, if for unknown reasons Splunk doesn't give you request_path, simply add an extract command and skip all the rex which is expensive.

Tags (1)
0 Karma

Loves-to-Learn Everything

Thanks @yuanliu, I explained why I couldn't use path directly, because it contains actual parameters. 

For example, for the route /orders/{orderID}, the path could be:




I want to analyze, for example, count of failed requests, or percentiles of call duration on this particular API route /orders/{orderID}.  

Of course I can modify my service code to print the route pattern in log, but that is another way, i need to deploy new code to production environment. 

0 Karma


Have you tried my previous code?

| eval route = if(match(request_path, "^/orders/\d+"), "/order/{orderID}", null())

This does exactly what you ask: create a new field named route that has a fixed pattern "/order/{orderID}".  Is there anything wrong with this?

In fact, because you really only care about first segment of the path - that fixed string "{orderID}" is just a decoration, the command could be simplified to slightly less expensive

| eval route = "/" . mvindex(split(request_path, "/"), 1) . "/{orderID}"

You can do whatever analysis against this field.

0 Karma


Hi @codewarrior,

If I got it correct, your need is extract a new field named "route" and it will contain the value after "orders/". 

You can capture it in your rex command, please try below;

level=info request.elapsed=(?<duration>.*) request.method=(?<method>.*) request.path=(?<path>.+orders\/(?<route>.+)) request_id=(?<request_id>.*) response.status=(?<statusCode>.*)


If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

Loves-to-Learn Everything

Thanks @scelikok. No I don't just want the orderID. But I want to manually create the RESTful API routing pattern.

for "path=/order/123456",  "route=/order/{orderID}", basically I am trying to use regex to replace the value and create a new field in this way:

if value matches \/order\/\d{12}, then convert to /order/{orderID}

I have other examples like:





0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...