Splunk Search

persoas coeficient correlation calculate manually

LeRoiGanesh22
Loves-to-Learn

hello , 

i have a problem i want to calculate a persoas coefficient to do correlation by the loop but i have a big issue . i have more than 23 fields and do the calculation manually making waste time a lot and i have a big syntax. someone know how i can get loop result without using MLTOOKIT. 


| fields TotalCount, usages, licenseb, StorageMb, Role_number, Siglum_number, SourceTypeDescription_number *_number
| eval sq_TotalCount = TotalCount * TotalCount
| eval sq_usages = usages * usages
| eval sq_licenseb = licenseb * licenseb
| eval sq_StorageMb = StorageMb * StorageMb
| eval sq_Role_number = Role_number * Role_number
| eval sq_Siglum_number = Siglum_number * Siglum_number
| eval sq_SourceTypeDescription_number = SourceTypeDescription_number * SourceTypeDescription_number

| eval product_TotalCount_usages = TotalCount * usages
| eval product_TotalCount_licenseb = TotalCount * licenseb
| eval product_TotalCount_StorageMb = TotalCount * StorageMb
| eval product_TotalCount_Role_number = TotalCount * Role_number
| eval product_TotalCount_Siglum_number = TotalCount * Siglum_number
| eval product_TotalCount_SourceTypeDescription_number = TotalCount * SourceTypeDescription_number

| stats sum(TotalCount) as sum_TotalCount, sum(sq_TotalCount) as sum_sq_TotalCount,
sum(usages) as sum_usages, sum(sq_usages) as sum_sq_usages,
sum(licenseb) as sum_licenseb, sum(sq_licenseb) as sum_sq_licenseb,
sum(StorageMb) as sum_StorageMb, sum(sq_StorageMb) as sum_sq_StorageMb,
sum(Role_number) as sum_Role_number, sum(sq_Role_number) as sum_sq_Role_number,
sum(Siglum_number) as sum_Siglum_number, sum(sq_Siglum_number) as sum_sq_Siglum_number,
sum(SourceTypeDescription_number) as sum_SourceTypeDescription_number, sum(sq_SourceTypeDescription_number) as sum_sq_SourceTypeDescription_number

sum(product_TotalCount_usages) as sum_TotalCount_usages,
sum(product_TotalCount_licenseb) as sum_TotalCount_licenseb,
sum(product_TotalCount_StorageMb) as sum_TotalCount_StorageMb,
sum(product_TotalCount_Role_number) as sum_TotalCount_Role_number,
sum(product_TotalCount_Siglum_number) as sum_TotalCount_Siglum_number,
sum(product_TotalCount_SourceTypeDescription_number) as sum_TotalCount_SourceTypeDescription_number

count as count

| eval pearson_TotalCount_usages = ((count * sum_TotalCount_usages) - (sum_TotalCount * sum_usages)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_usages) - (sum_usages * sum_usages))),
pearson_TotalCount_licenseb = ((count * sum_TotalCount_licenseb) - (sum_TotalCount * sum_licenseb)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_licenseb) - (sum_licenseb * sum_licenseb))),
pearson_TotalCount_StorageMb = ((count * sum_TotalCount_StorageMb) - (sum_TotalCount * sum_StorageMb)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_StorageMb) - (sum_StorageMb * sum_StorageMb))),
pearson_TotalCount_Role_number = ((count * sum_TotalCount_Role_number) - (sum_TotalCount * sum_Role_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_Role_number) - (sum_Role_number * sum_Role_number))),
pearson_TotalCount_Siglum_number = ((count * sum_TotalCount_Siglum_number) - (sum_TotalCount * sum_Siglum_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_Siglum_number) - (sum_Siglum_number * sum_Siglum_number))),
pearson_TotalCount_SourceTypeDescription_number = ((count * sum_TotalCount_SourceTypeDescription_number) - (sum_TotalCount * sum_SourceTypeDescription_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_SourceTypeDescription_number) - (sum_SourceTypeDescription_number * sum_SourceTypeDescription_number)))

| table pearson_TotalCount_usages, pearson_TotalCount_licenseb, pearson_TotalCount_StorageMb, pearson_TotalCount_Role_number, pearson_TotalCount_Siglum_number, pearson_TotalCount_SourceTypeDescription_number

Labels (2)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

I am not sure what you are asking for - you seem to be calculating Pearson's coefficient with TotalCount being one of the variables. Is this always the case? In any event, have you considered using macros to speed up writing the calculations out?

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...