Splunk Search

persoas coeficient correlation calculate manually

LeRoiGanesh22
Loves-to-Learn

hello , 

i have a problem i want to calculate a persoas coefficient to do correlation by the loop but i have a big issue . i have more than 23 fields and do the calculation manually making waste time a lot and i have a big syntax. someone know how i can get loop result without using MLTOOKIT. 


| fields TotalCount, usages, licenseb, StorageMb, Role_number, Siglum_number, SourceTypeDescription_number *_number
| eval sq_TotalCount = TotalCount * TotalCount
| eval sq_usages = usages * usages
| eval sq_licenseb = licenseb * licenseb
| eval sq_StorageMb = StorageMb * StorageMb
| eval sq_Role_number = Role_number * Role_number
| eval sq_Siglum_number = Siglum_number * Siglum_number
| eval sq_SourceTypeDescription_number = SourceTypeDescription_number * SourceTypeDescription_number

| eval product_TotalCount_usages = TotalCount * usages
| eval product_TotalCount_licenseb = TotalCount * licenseb
| eval product_TotalCount_StorageMb = TotalCount * StorageMb
| eval product_TotalCount_Role_number = TotalCount * Role_number
| eval product_TotalCount_Siglum_number = TotalCount * Siglum_number
| eval product_TotalCount_SourceTypeDescription_number = TotalCount * SourceTypeDescription_number

| stats sum(TotalCount) as sum_TotalCount, sum(sq_TotalCount) as sum_sq_TotalCount,
sum(usages) as sum_usages, sum(sq_usages) as sum_sq_usages,
sum(licenseb) as sum_licenseb, sum(sq_licenseb) as sum_sq_licenseb,
sum(StorageMb) as sum_StorageMb, sum(sq_StorageMb) as sum_sq_StorageMb,
sum(Role_number) as sum_Role_number, sum(sq_Role_number) as sum_sq_Role_number,
sum(Siglum_number) as sum_Siglum_number, sum(sq_Siglum_number) as sum_sq_Siglum_number,
sum(SourceTypeDescription_number) as sum_SourceTypeDescription_number, sum(sq_SourceTypeDescription_number) as sum_sq_SourceTypeDescription_number

sum(product_TotalCount_usages) as sum_TotalCount_usages,
sum(product_TotalCount_licenseb) as sum_TotalCount_licenseb,
sum(product_TotalCount_StorageMb) as sum_TotalCount_StorageMb,
sum(product_TotalCount_Role_number) as sum_TotalCount_Role_number,
sum(product_TotalCount_Siglum_number) as sum_TotalCount_Siglum_number,
sum(product_TotalCount_SourceTypeDescription_number) as sum_TotalCount_SourceTypeDescription_number

count as count

| eval pearson_TotalCount_usages = ((count * sum_TotalCount_usages) - (sum_TotalCount * sum_usages)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_usages) - (sum_usages * sum_usages))),
pearson_TotalCount_licenseb = ((count * sum_TotalCount_licenseb) - (sum_TotalCount * sum_licenseb)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_licenseb) - (sum_licenseb * sum_licenseb))),
pearson_TotalCount_StorageMb = ((count * sum_TotalCount_StorageMb) - (sum_TotalCount * sum_StorageMb)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_StorageMb) - (sum_StorageMb * sum_StorageMb))),
pearson_TotalCount_Role_number = ((count * sum_TotalCount_Role_number) - (sum_TotalCount * sum_Role_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_Role_number) - (sum_Role_number * sum_Role_number))),
pearson_TotalCount_Siglum_number = ((count * sum_TotalCount_Siglum_number) - (sum_TotalCount * sum_Siglum_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_Siglum_number) - (sum_Siglum_number * sum_Siglum_number))),
pearson_TotalCount_SourceTypeDescription_number = ((count * sum_TotalCount_SourceTypeDescription_number) - (sum_TotalCount * sum_SourceTypeDescription_number)) / (sqrt((count * sum_sq_TotalCount) - (sum_TotalCount * sum_TotalCount)) * sqrt((count * sum_sq_SourceTypeDescription_number) - (sum_SourceTypeDescription_number * sum_SourceTypeDescription_number)))

| table pearson_TotalCount_usages, pearson_TotalCount_licenseb, pearson_TotalCount_StorageMb, pearson_TotalCount_Role_number, pearson_TotalCount_Siglum_number, pearson_TotalCount_SourceTypeDescription_number

Labels (2)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

I am not sure what you are asking for - you seem to be calculating Pearson's coefficient with TotalCount being one of the variables. Is this always the case? In any event, have you considered using macros to speed up writing the calculations out?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...