Splunk Search

Fix replace command

Explorer

Background

I have a log file where I have extracted some fields. I am trying to parse a field to get the numeric values it has using replace but it is not working and I don't understand why.

Problem

I have a long log file and one of the fields I extracted is called metrics_total and has the following format: "Total: __decimal_number__", where decimal number is any floating point number.

My objective is to create an average of this field, but because I have the string "Total: " the avg command fails. So I am trying to remove it using replace. However I am failing.

Query

This is how I am trying to use replace:


host=host00 OR host01 endpoint=* http_method=* http_status=200 metrics_total=* | replace "Total: " with "" in metrics_total | table http_method endpoint metrics_total

Where host, endpoint, http_method, http_status and metrics_total are extracted fields.

The issue here is that no matter what I do, nothing changes. This is what I get:


GET /product/bananas Total: 0.087
GET /product/apples Total: 0.003
GET /cart/checkout Total: 0.005

And this is what I actually want to achieve:


GET /product/bananas 0.087
GET /product/apples 0.003
GET /cart/checkout 0.005

Here I would get only the numbers instead of the whole Total: 0.087 string.

Going further

Going even further I would really like to have this field computed into an average. As in, the avg(metrics_total)for each endpoint grouped by http_method.

Questions

  1. What is wrong in my usage of replace?
  2. How can I compute the average metric for each endpoint grouped by http_method?
  3. Is there an easier way to achieve my objective? (Am I complicating things too much?)
0 Karma
1 Solution

SplunkTrust
SplunkTrust

@pedroma,

Try

"Your search"
|rex field=metrics_total "(?<Total>\d+.\d+)"
|stats avg(Total) as avg_by_method by http_method

replace works well with full string replacements

View solution in original post

Esteemed Legend

I am not sure what is wrong but just do it another way, like this:

| rex field=metrics_total mode=sed "s/Total:\s*//"
0 Karma

SplunkTrust
SplunkTrust

@pedroma,

Try

"Your search"
|rex field=metrics_total "(?<Total>\d+.\d+)"
|stats avg(Total) as avg_by_method by http_method

replace works well with full string replacements

View solution in original post

Explorer

Thanks for the query! Could you explain me why my replace was incorrect and why using rex was better for my use case?

I don't quite understand what you mean with "full string replacements".

0 Karma