What is the "correlate" function doing behind the scenes

[spogtrop]

I am trying to use the correlate command in Splunk but keep receiving "1.0" or other numbers as the correlation value when it should not. For example, I have two columns in my table, each with values "increase" or "decrease" based on how much data it is ingesting hour to hour. When I use correlate after that, however, I get 1.0 as the correlation value when it is not 100%. So what exactly is the command correlating, is it not the table? Is it something with the indexes behind the scenes? Also, how do you use parentheses after the correlate command to input fields? All help is appreciated, I have been working on this for a while.

[richgalloway]

The correlate command looks at the fields in your data and generates a matrix showing the percentage of times each pair of fields appears.

You don't use parentheses or any other arguments with the correlate command.  If you need to specify fields, use the contingency command.

See https://docs.splunk.com/Documentation/Splunk/9.0.0/SearchReference/Correlate and https://docs.splunk.com/Documentation/Splunk/9.0.0/SearchReference/Contingency for more information.

[spogtrop]

Thank you, so then can I only use the correlate command with built in fields? Or can I do what I did, create to columns using "eval" and then compare them? I am basically trying to run a correlation the way Microsoft Excel would, where you take two columns with numbers and run the correlation. Even though my data is different, I get values like 1.0 which can't be right because the data is not 100% the same. 

[richgalloway · Accepted solution]

The correlate command uses all fields in your data that have names not beginning with an underscore.

The correlate command is very different from Excel's CORREL function.  The latter compares values whereas the former checks for the *presence* of values (ignoring the actual value).

As far as I can tell, there is no command or function similar to CORREL in Splunk.

[spogtrop]

That was a lot of help thank you so much