Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Splunk Search

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Splunk Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: Running multiple macros in the same search

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Raekmackbert

Explorer

06-29-2022
03:58 AM

Hi all!

I'm trying to run multiple macros in the same search and eventually aggregate the results from each execution into a table.

My current search looks like this, which seems to work fine for a single execution of the histperc macro (Prometheus integration provided)

```
| mstats rate(_value) AS requests WHERE "index"="MyIndex" AND metric_name="MyMetricNameRegex" BY metric_name, le
| stats sum(requests) AS total_requests BY metric_name, le
| `histperc(0.5, total_requests, le, metric_name)`
| rename histperc as Median
| table metric_name Median 90th 75th 25th 10th
```

I think the issue is that total_requests value is not passed down after the | `histperc(0.5, total_requests, le, metric_name)` row but i am not sure if this is the case. Also not sure if rename is by reference or copy and what would eventually happen by having many renames and overrides of the histperc value like below.

The histperc macro looks like this:

```
sort $groupby$, $le$
| eventstats max($hist_rate$) as total_hist_rate, last($le$) as uppermost_bound, count as num_buckets by $groupby$
| eval rank=exact($perc$)*total_hist_rate
| streamstats current=f last($le$) as gr, last($hist_rate$) as last_hist_rate by $groupby$
| eval gr=if(isnull(gr), 0, gr), last_hist_rate=if(isnull(last_hist_rate), 0, last_hist_rate)
| where $hist_rate$ >= rank
| dedup $groupby$
| eval res=case(lower(uppermost_bound) != "+inf" or num_buckets < 2, "NaN", lower($le$) == "+inf", gr, gr == 0 and $le$ <= 0, $le$, true(), exact(gr + ($le$-gr)*(rank - last_hist_rate) / ($hist_rate$ - last_hist_rate)))
| fields $groupby$, res
| rename res as "histperc"
```

What i want to do is something like this:

```
| mstats rate(_value) AS requests WHERE "index"="MyIndex" AND metric_name="MyMetricNameRegex" BY metric_name, le
| stats sum(requests) AS total_requests BY metric_name, le
| `histperc(0.5, total_requests, le, metric_name)`
| rename histperc as Median
| `histperc(0.9, total_requests, le, metric_name)`
| rename histperc as 90th
| `histperc(0.1, total_requests, le, metric_name)`
| rename histperc as 10th
| `histperc(0.75, total_requests, le, metric_name)`
| rename histperc as 75th
| `histperc(0.25, total_requests, le, metric_name)`
| rename histperc as 25th
| table metric_name Median 90th 75th 25th 10th
```

Thankful for all help!

1 Solution

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Raekmackbert

Explorer

06-29-2022
07:11 AM

I found a solution that seems to work for me.. The issue was in the histperc macro when attempting to overwrite already created fields with new values. The query i came up with is

```
| mstats rate(_value) AS requests WHERE "index"="MyIndex" AND metric_name="MyMetricNameRegex" BY metric_name, le
| stats sum(requests) AS total_requests BY metric_name, le
| sort metric_name, le
| eventstats max(total_requests) as total_hist_rate, last(le) as uppermost_bound, count as num_buckets by metric_name
| eval rank=exact(0.5)*total_hist_rate
| streamstats current=f last(le) as gr, last(total_requests) as last_hist_rate by metric_name
| dedup metric_name
| `histperc_partial(Median, 0.5, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(90th, 0.9, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(75th, 0.75, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(25th, 0.25, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(10th, 0.10, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| table metric_name Median 90th 75th 25th 10th
```

I had to create a new macro that does what the original histperc macro does but where fields are already calculated beforehand, the macro looks like

```
eval rank=exact($hist_perc$)*$total_hist_rate$
| eval $gr$=if(isnull($gr$), 0, $gr$), $last_hist_rate$=if(isnull($last_hist_rate$), 0, $last_hist_rate$)
| where $total_requests$ >= rank
| eval $fieldName$=case(lower($uppermost_bound$) != "+inf" or $num_buckets$ < 2, "NaN", lower($le$) == "+inf", $gr$, $gr$ == 0 and $le$ <= 0, $le$, true(), exact($gr$ + ($le$-$gr$)*(rank - $last_hist_rate$) / ($total_requests$ - $last_hist_rate$)))
```

Im sure there are ways to make this alot simpler and faster than what i have done, but this solved the issue for me at the moment. If anyone has ideas of where to improve i will gladly accept them, but for now i'll mark this question as answered

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Raekmackbert

Explorer

06-29-2022
07:11 AM

I found a solution that seems to work for me.. The issue was in the histperc macro when attempting to overwrite already created fields with new values. The query i came up with is

```
| mstats rate(_value) AS requests WHERE "index"="MyIndex" AND metric_name="MyMetricNameRegex" BY metric_name, le
| stats sum(requests) AS total_requests BY metric_name, le
| sort metric_name, le
| eventstats max(total_requests) as total_hist_rate, last(le) as uppermost_bound, count as num_buckets by metric_name
| eval rank=exact(0.5)*total_hist_rate
| streamstats current=f last(le) as gr, last(total_requests) as last_hist_rate by metric_name
| dedup metric_name
| `histperc_partial(Median, 0.5, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(90th, 0.9, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(75th, 0.75, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(25th, 0.25, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| `histperc_partial(10th, 0.10, total_requests, total_hist_rate, gr, last_hit_rate, num_buckets, uppermost_bound, le)`
| table metric_name Median 90th 75th 25th 10th
```

I had to create a new macro that does what the original histperc macro does but where fields are already calculated beforehand, the macro looks like

```
eval rank=exact($hist_perc$)*$total_hist_rate$
| eval $gr$=if(isnull($gr$), 0, $gr$), $last_hist_rate$=if(isnull($last_hist_rate$), 0, $last_hist_rate$)
| where $total_requests$ >= rank
| eval $fieldName$=case(lower($uppermost_bound$) != "+inf" or $num_buckets$ < 2, "NaN", lower($le$) == "+inf", $gr$, $gr$ == 0 and $le$ <= 0, $le$, true(), exact($gr$ + ($le$-$gr$)*(rank - $last_hist_rate$) / ($total_requests$ - $last_hist_rate$)))
```

Im sure there are ways to make this alot simpler and faster than what i have done, but this solved the issue for me at the moment. If anyone has ideas of where to improve i will gladly accept them, but for now i'll mark this question as answered

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smurf

Communicator

06-29-2022
05:34 AM

Hi,

I think the issue is that you are renaming the filed and using the original field again. When you rename it, the original field does not exist anymore, so in the next macro call you need to use the new field name.

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Raekmackbert

Explorer

06-29-2022
05:37 AM

Hi!

The histperc field is created by the execution of the macro, so wouldn't running the macro many times in sequence just recreate/update the field with a new value each time.

I updated the original post with the actual macro as well in case you would like to see it (Even though the macro itself isnt mine)

Get Updates on the Splunk Community!

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2308!
Analysts can ...

November 2023 Edition
Hayyy Splunk Education Enthusiasts and the Eternally Curious!
We’re back with another ...

For the past three years, Splunk has partnered with Enterprise Strategy Group to conduct a survey that gauges ...