How to generate statistical data without knowing field names in advance?

decenior — Fri, 18 Feb 2022 01:09:06 GMT

Honored Splunkodes,

I am trying to keep track of the manpower in each of my legions, so that if any legion loses too many troops at once, I know which one to reinforce.
However, I have many legions, and thus I track all of their manpower without knowing which ones will be important each day. I can't leave my myrmidons without reinforcements!

I'd like to generate statistical information about them at the time of graph generation.

Currently I'm doing this, it's dirty but it works.

I get my legion manpower by querying that index, dropping any that don't fall in the top 50.
index=legions LegionName=*
| timechart span=1d limit=50 count by LegionName
| fields - OTHER
| untable _time LegionName ManPower
| outputlookup append=f mediterranean_legions.csv

This gives me a quick idea which legions have lost (or gained) a lot of manpower each day.

Now ideally, I'd like to generate standard deviation and determine if they are outliers based on z score rather than just guessing with the lower and upper bound values.

If this worked, I'd get what I want. Is there a way to accomplish this?

| streamstats global=f window=10 avg(ManPower) AS mp_avg by LegionName, stdev(ManPower) as mp_stdev by LegionName, max(ManPower) as mp_max by LegionName, min(ManPower) as mp_min by LegionName

Re: How to generate statistical data without knowing field names in advance?

ITWhisperer — Fri, 18 Feb 2022 08:06:40 GMT

I am not sure what it is you are trying to accomplish but does something like this work?

| gentimes start=-28 increment=1m | rename starttime as _time | streamstats count as row | eval LegionName=mvindex(split("ABCDEFGHIJKLMNOPQRSTUVWXYZ",""),random()%4).mvindex(split("ABCDEFGHIJKLMNOPQRSTUVWXYZ",""),random()%25) | timechart span=1d limit=50 count by LegionName | fields - OTHER | untable _time LegionName ManPower | timechart avg(ManPower) by LegionName | fields - OTHER | untable _time LegionName ManPower | streamstats global=f window=10 avg(ManPower) AS mp_avg stdev(ManPower) as mp_stdev max(ManPower) as mp_max min(ManPower) as mp_min by LegionName | eval lowerBound=mp_avg-mp_stdev | eval upperBound=mp_avg+mp_stdev | eval isOutlier=if('ManPower' < lowerBound OR 'ManPower' > upperBound, "XXX".ManPower, ManPower) | search isOutlier="XXX*" | table _time, LegionName, ManPower, *

topic How to generate statistical data without knowing field names in advance? in Splunk Search

How to generate statistical data without knowing field names in advance?

Re: How to generate statistical data without knowing field names in advance?