Splunk Search

How to perform stats on large number of fields matching a certain pattern without doing stats on each one individually?

mitag
Contributor

How do I perform stats on a large number of fields matching a certain pattern without doing stats on each one individually? In a sample event below, there are 10+ fields with names beginning with "er_". My task is to fire an alert if any of the values in these fields increases from the previous event.

Sample event:

 

   er_bad_eof: 0
   er_bad_os: 0
   er_crc: 0
   er_crc_good_eof: 0
   er_enc_in: 0
   er_enc_out: 0
   er_inv_arb: 0
   er_lun_zone_miss: 0
   er_multi_credit_loss: 0
   er_other_discard: 11
   er_pcs_blk: 0
   er_rx_c3_timeout: 0
   er_single_credit_loss: 0
   er_toolong: 0
   er_trunc: 0
   er_tx_c3_timeout: 0
   er_type1_miss: 0
   er_type2_miss: 0
   er_type6_miss: 0
   er_unreachable: 0
   er_unroutable: 11
   er_zone_miss: 0
   lgc_stats_clear_ts: Never
   phy_stats_clear_ts: Never
   port_description: slot12 port46
   port_name: 382

 

SPL where I run stats on just two of those fields and where the "er_..._delta" values will be used to fire an alert if they're > 0:

 

index="sandbox" source="HEC"

| stats count AS events,
        min(er_enc_out) AS er_enc_out_min,
        max(er_enc_out) AS er_enc_out_max,
        min(er_other_discard) AS er_other_discard_min,
        max(er_other_discard) AS er_other_discard_max,
        by host port_name, port_description
| eval er_enc_out_delta = er_enc_out_max-er_enc_out_min,
       er_other_discard_delta = er_other_discard_max - er_other_discard_min
| sort -er_enc_out_delta -er_other_discard_delta -er_enc_out_max -er_other_discard_max port_name

 

How do I run similar stats on all fields with names beginning with "er_"?

Thanks!

Labels (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

You can use wildcards in stats.

| stats max(er_*) as er_*_max

 

---
If this reply helps you, Karma would be appreciated.

mitag
Contributor

Thanks @richgalloway !

What about evaluating the delta (max - min)? Wildcards don't seem to work there. (Never mind this - sounds like I'd just use the range stats function.)

To fire an alert, I'd need to eval the condition of range(er_*) > 0 and then expose the offending fields. Yet the search er_*_range > 0 part in the below doesn't seem to work.

 

| stats count AS events,
        min(er_*) AS er_*_min,
        max(er_*) AS er_*_max,
        range(er_*) AS er_*_range,
        by host port_name, port_description
| fields host port_name port_description events er_*_range 
| search er_*_range > 0
| sort -er_*_range port_name

 

How do I do this? (Check if any of the er_*_range field values are positive?) Something like this?

| stats count AS events,
        range(er_*) AS er_*_range,
        by host port_name, port_description

| foreach er_*_range 
    [eval item='<<FIELD>>']
| where item != 0 

| fields host port_name port_description events er_*_range 

| sort -er_*_range port_name
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, that's the approach I would take, but haven't tested it.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...