All Apps and Add-ons

Splunk For loop

arun_kant_sharm
Path Finder

Hi Experts,

Below is the JSON format input of my data, I want to fetch LoadBalancer name from metric_dimensions fields, but the position of Load balancer is differ in both field.

I don't know how to create for loop with break in SPL, please suggest how I achieve this.

{ [-]
Average: 0.5441528732026144
Maximum: 14.997758
Minimum: 0.000371
SampleCount: 1530
Sum: 832.553896
Unit: Seconds
account_id: 522995424064
metric_dimensions: LoadBalancer=[app/nonprod-web-in-alb/bXXXXXXXXXXXXXX]
metric_name: TargetResponseTime
period: 300
timestamp: 2019-12-05T01:25:00Z
}

{ [-]
Average: 0.6173158354037267
Maximum: 10.601669
Minimum: 0.000397
SampleCount: 644
Sum: 397.551398
Unit: Seconds
account_id: 522995424064
metric_dimensions: AvailabilityZone=[ap-northeast-1d],LoadBalancer=[app/nonprod-web-in-alb/bXXXXXXXXXXXXXX],TargetGroup=[targetgroup/preprod-portals-perf-443/aXXXXXXXXXXXXX]
metric_name: TargetResponseTime
period: 300
timestamp: 2019-12-05T01:25:00Z
}

Tags (2)
0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval raw="[{\"Average\": 0.5441528732026144,
\"Maximum\": 14.997758,
\"Minimum\": 0.000371,
\"SampleCount\": 1530,
\"Sum\": 832.553896,
\"Unit\": \"Seconds\",
\"account_id\": 522995424064,
\"metric_dimensions\": \"LoadBalancer=[app/nonprod-web-in-alb/bXXXXXXXXXXXXXX]\",
\"metric_name\": \"TargetResponseTime\",
\"period\": 300,
\"timestamp\": \"2019-12-05T01:25:00Z\"},
{\"Average\": 0.6173158354037267,
\"Maximum\": 10.601669,
\"Minimum\": 0.000397,
\"SampleCount\": 644,
\"Sum\": 397.551398,
\"Unit\": \"Seconds\",
\"account_id\": 522995424064,
\"metric_dimensions\": \"AvailabilityZone=[ap-northeast-1d],LoadBalancer=[app/nonprod-web-in-alb/bXXXXXXXXXXXXXX],TargetGroup=[targetgroup/preprod-portals-perf-443/aXXXXXXXXXXXXX]\"
\"metric_name\": \"TargetResponseTime\",
\"period\": 300,
\"timestamp\": \"2019-12-05T01:25:00Z\"}]"
| spath input=raw 
| mvexpand {}.Average
| streamstats count
| foreach {}.*
    [| rename <<FIELD>> as <<MATCHSTR>>
    | eval <<MATCHSTR>> = if(mvcount('<<MATCHSTR>>')=1,'<<MATCHSTR>>',mvindex('<<MATCHSTR>>',count - 1))   ]
| eval _raw=metric_dimensions
| kv
| fields - _* , raw
| eval source="ApplicationELB"
| table source metric_dimensions LoadBalancer Average Unit

Hi, folks.
streamstats and foreach are useful.

woodcock
Esteemed Legend

I just bookmarked it.

0 Karma

to4kawa
Ultra Champion

Thank you very much @woodcock.

I look forward to working with you.

0 Karma

woodcock
Esteemed Legend

Any time. What do you have cooking?

0 Karma

to4kawa
Ultra Champion

JSON etc...

0 Karma

woodcock
Esteemed Legend

This is a GREAT answer.

0 Karma

woodcock
Esteemed Legend

You should be able to add | spath to your search and get all of your fields (also try eval foo=spath()) but if it is not valid JSON, try this:

| makeresults 
|  eval raw="Fee Fie Fo Fum {Average: 0.5441528732026144
Maximum: 14.997758
Minimum: 0.000371
SampleCount: 1530
Sum: 832.553896
Unit: Seconds
account_id: 522995424064
metric_dimensions: LoadBalancer=[app/nonprod-web-in-alb/bXXXXXXXXXXXXXX]
metric_name: TargetResponseTime
period: 300
timestamp: 2019-12-05T01:25:00Z
} foo bar bat:::Fee Fie Fo Fum {Average: 0.6173158354037267
Maximum: 10.601669
Minimum: 0.000397
SampleCount: 644
Sum: 397.551398
Unit: Seconds
account_id: 522995424064
metric_dimensions: AvailabilityZone=[ap-northeast-1d],LoadBalancer=[app/nonprod-web-in-alb/bXXXXXXXXXXXXXX],TargetGroup=[targetgroup/preprod-portals-perf-443/aXXXXXXXXXXXXX]
metric_name: TargetResponseTime
period: 300
timestamp: 2019-12-05T01:25:00Z
} foo bar bat"
| makemv delim=":::" raw
| mvexpand raw
| rename raw AS _raw

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"
| rex mode=sed "s/.*{Average:/Average:{/ s/}.*/}/"
| kv
0 Karma

arun_kant_sharm
Path Finder

This is a valid JSON, generated by AWS Cloudwatch.

I am using the below SPL:
sourcetype=aws:cloudwatch
| spath path=SampleCount
| spath path=metric_dimensions
| spath path=metric_name
| spath path=timestampe
| search source = "*ApplicationELB" AND metric_name= TargetResponseTime | where Average > 0.3 | eval LoadBalancer = mvindex(split(metric_dimensions,","), 1) | table source metric_dimensions LoadBalancer Average Unit

But using above SPL LoadBalancer are populate empty for some events, because I pass 1 in mvindex, so do you now any way to iterate in the output of split function ?

0 Karma

vmacedo
Explorer

@arun_kant_sharma, instead of using mvindex/split use split to create a multivalue field and mvfilter to get the LoadBalancer wherever it is:

sourcetype=aws:cloudwatch
| spath path=SampleCount
| spath path=metric_dimensions
| spath path=metric_name
| spath path=timestampe
| search source = "*ApplicationELB" AND metric_name= TargetResponseTime | where Average > 0.3 | eval LoadBalancer = mvfilter(match(split(metric_dimensions,","),"LoadBalancer") | table source metric_dimensions LoadBalancer Average Unit

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...