Splunk Search

persoas coeficient correlation calculate manually

LeRoiGanesh22
Loves-to-Learn

hello , 

i have a problem i want to calculate a persoas coefficient to do correlation by the loop but i have a big issue . i have more than 23 fields and do the calculation manually making waste time a lot and i have a big syntax. someone know how i can get loop result without using MLTOOKIT. 


| fields TotalCount, usages, licenseb, StorageMb, Role_number, Siglum_number, SourceTypeDescription_number *_number
| eval sq_TotalCount = TotalCount * TotalCount
| eval sq_usages = usages * usages
| eval sq_licenseb = licenseb * licenseb
| eval sq_StorageMb = StorageMb * StorageMb
| eval sq_Role_number = Role_number * Role_number
| eval sq_Siglum_number = Siglum_number * Siglum_number
| eval sq_SourceTypeDescription_number = SourceTypeDescription_number * SourceTypeDescription_number

| eval product_TotalCount_usages = TotalCount * usages
| eval product_TotalCount_licenseb = TotalCount * licenseb
| eval product_TotalCount_StorageMb = TotalCount * StorageMb
| eval product_TotalCount_Role_number = TotalCount * Role_number
| eval product_TotalCount_Siglum_number = TotalCount * Siglum_number
| eval product_TotalCount_SourceTypeDescription_number = TotalCount * SourceTypeDescription_number

| stats sum(TotalCount) as sum_TotalCount, sum(sq_TotalCount) as sum_sq_TotalCount,
sum(usages) as sum_usages, sum(sq_usages) as sum_sq_usages,
sum(licenseb) as sum_licenseb, sum(sq_licenseb) as sum_sq_licenseb,
sum(StorageMb) as sum_StorageMb, sum(sq_StorageMb) as sum_sq_StorageMb,
sum(Role_number) as sum_Role_number, sum(sq_Role_number) as sum_sq_Role_number,
sum(Siglum_number) as sum_Siglum_number, sum(sq_Siglum_number) as sum_sq_Siglum_number,
sum(SourceTypeDescription_number) as sum_SourceTypeDescription_number, sum(sq_SourceTypeDescription_number) as sum_sq_SourceTypeDescription_number

sum(product_TotalCount_usages) as sum_TotalCount_usages,
sum(product_TotalCount_licenseb) as sum_TotalCount_licenseb,
sum(product_TotalCount_StorageMb) as sum_TotalCount_StorageMb,
sum(product_TotalCount_Role_number) as sum_TotalCount_Role_number,
sum(product_TotalCount_Siglum_number) as sum_TotalCount_Siglum_number,
sum(product_TotalCount_SourceTypeDescription_number) as sum_TotalCount_SourceTypeDescription_number

count as count

| eval pearson_TotalCount_usages = ((count * sum_TotalCount_usages) - (sum_TotalCount * sum_usages)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_usages) - (sum_usages * sum_usages))),
pearson_TotalCount_licenseb = ((count * sum_TotalCount_licenseb) - (sum_TotalCount * sum_licenseb)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_licenseb) - (sum_licenseb * sum_licenseb))),
pearson_TotalCount_StorageMb = ((count * sum_TotalCount_StorageMb) - (sum_TotalCount * sum_StorageMb)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_StorageMb) - (sum_StorageMb * sum_StorageMb))),
pearson_TotalCount_Role_number = ((count * sum_TotalCount_Role_number) - (sum_TotalCount * sum_Role_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_Role_number) - (sum_Role_number * sum_Role_number))),
pearson_TotalCount_Siglum_number = ((count * sum_TotalCount_Siglum_number) - (sum_TotalCount * sum_Siglum_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_Siglum_number) - (sum_Siglum_number * sum_Siglum_number))),
pearson_TotalCount_SourceTypeDescription_number = ((count * sum_TotalCount_SourceTypeDescription_number) - (sum_TotalCount * sum_SourceTypeDescription_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_SourceTypeDescription_number) - (sum_SourceTypeDescription_number * sum_SourceTypeDescription_number)))

| table pearson_TotalCount_usages, pearson_TotalCount_licenseb, pearson_TotalCount_StorageMb, pearson_TotalCount_Role_number, pearson_TotalCount_Siglum_number, pearson_TotalCount_SourceTypeDescription_number

Labels (2)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

I am not sure what you are asking for - you seem to be calculating Pearson's coefficient with TotalCount being one of the variables. Is this always the case? In any event, have you considered using macros to speed up writing the calculations out?

0 Karma
Get Updates on the Splunk Community!

Index This | What’s a riddle wrapped in an enigma?

September 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

BORE at .conf25

Boss Of Regular Expression (BORE) was an interactive session run again this year at .conf25 by the brilliant ...

OpenTelemetry for Legacy Apps? Yes, You Can!

This article is a follow-up to my previous article posted on the OpenTelemetry Blog, "Your Critical Legacy App ...