Splunk Search

Fix replace command

pedroma
Engager

Background

I have a log file where I have extracted some fields. I am trying to parse a field to get the numeric values it has using replace but it is not working and I don't understand why.

Problem

I have a long log file and one of the fields I extracted is called metrics_total and has the following format: "Total: __decimal_number__", where decimal number is any floating point number.

My objective is to create an average of this field, but because I have the string "Total: " the avg command fails. So I am trying to remove it using replace. However I am failing.

Query

This is how I am trying to use replace:


host=host00 OR host01 endpoint=* http_method=* http_status=200 metrics_total=* | replace "Total: " with "" in metrics_total | table http_method endpoint metrics_total

Where host, endpoint, http_method, http_status and metrics_total are extracted fields.

The issue here is that no matter what I do, nothing changes. This is what I get:


GET /product/bananas Total: 0.087
GET /product/apples Total: 0.003
GET /cart/checkout Total: 0.005

And this is what I actually want to achieve:


GET /product/bananas 0.087
GET /product/apples 0.003
GET /cart/checkout 0.005

Here I would get only the numbers instead of the whole Total: 0.087 string.

Going further

Going even further I would really like to have this field computed into an average. As in, the avg(metrics_total)for each endpoint grouped by http_method.

Questions

  1. What is wrong in my usage of replace?
  2. How can I compute the average metric for each endpoint grouped by http_method?
  3. Is there an easier way to achieve my objective? (Am I complicating things too much?)
0 Karma
1 Solution

renjith_nair
Legend

@pedroma,

Try

"Your search"
|rex field=metrics_total "(?<Total>\d+.\d+)"
|stats avg(Total) as avg_by_method by http_method

replace works well with full string replacements

---
What goes around comes around. If it helps, hit it with Karma 🙂

View solution in original post

woodcock
Esteemed Legend

I am not sure what is wrong but just do it another way, like this:

| rex field=metrics_total mode=sed "s/Total:\s*//"
0 Karma

renjith_nair
Legend

@pedroma,

Try

"Your search"
|rex field=metrics_total "(?<Total>\d+.\d+)"
|stats avg(Total) as avg_by_method by http_method

replace works well with full string replacements

---
What goes around comes around. If it helps, hit it with Karma 🙂

pedroma
Engager

Thanks for the query! Could you explain me why my replace was incorrect and why using rex was better for my use case?

I don't quite understand what you mean with "full string replacements".

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...