How do I compare two vector fields from my log and produce an output?

[ap666]

I have a field called start.point and end.point in my logs. We can assume it has values in x and y coordinates.

A part of my raw logs looks like this: 

... , "start" : { "time_s" : 1234 , "point" : [2.5,5.5]}, "end" : { "time_s" : 2344 , "point" : [9.5,8.5]}, ...

And in the list view, when I select the fields it shows in this format: 

start.point{} = 2.5 start.point{} = 5.5 end.point{} = 9.5 end.point{} = 2.5 

Now, all I want to do is calculate the distance from start point to the end point and construct a dasbboard that shows the distance graph. The formula is: √[(x2 – x1)^2 + (y2 – y1)^2]. However, I am having difficulty extracting the values x1, y1 and x2, y2 from the field start.point and end.point. 

[yuanliu · Accepted solution]

As you indicated, start.point{} and end.point{} already exist, and are vectors.  There is no need to "extract", much less using regex. (Do not treat structured data as text, as I always say.)

In your data, a vector is represented as a JSON array.  In SPL, an array is denoted as field{}; you can access its indexed components by mvindex.  Add Pitagora's theorem to this mix, you get

| eval distance = sqrt(pow(tonumber(mvindex('end.point{}', 0)) - tonumber(mvindex('start.point{}', 0)), 2) + pow(tonumber(mvindex('end.point{}', 1)) - tonumber(mvindex('start.point{}', 1)), 2))

[ap666]

Thank you so much. It worked.

[thevikramyadav]

One way to do this is by using Splunk's rex command, which allows you to extract fields using regular expressions. Here's an example search query that extracts the x and y values for start.point and end.point and calculates the distance

your_search_here | rex field=start.point "\[(?<x1>\d+\.\d+),(?<y1>\d+\.\d+)\]"
| rex field=end.point "\[(?<x2>\d+\.\d+),(?<y2>\d+\.\d+)\]"
| eval distance = sqrt(pow((x2 - x1), 2) + pow((y2 - y1), 2))

[aoverfield]

You can regex the fields out.

| makeresults 
| eval start="{ \"time_s\" : 1234 , \"point\" : [2.5,5.5]}", end="{ \"time_s\" : 2344 , \"point\" : [9.5,8.5]}" 
| rex field=start "\"point\"\s\:\s\[(?<x1>\d+\.\d+)\,(?<x2>\d+\.\d+)\]" 
| rex field=end "\"point\"\s\:\s\[(?<y1>\d+\.\d+)\,(?<y2>\d+\.\d+)\]" 
| eval result = sqrt( pow((x2 - x1),2) + pow((y2 - y1),2) )

[ap666]

It did not work. It says I have to "use makeresults as a first argument"

To be more clear, I have bunch of query in the front already to filter specific logs that I am interested. I have to now extract the coordinates from the field start.point and end.point, perform the math and display the result. 

[aoverfield]

The | makeresults were a test. Only add in the regex portion to your search.

<Initial Portion of your search>
| rex field=start "\"point\"\s\:\s\[(?<x1>\d+\.\d+)\,(?<x2>\d+\.\d+)\]" 
| rex field=end "\"point\"\s\:\s\[(?<y1>\d+\.\d+)\,(?<y2>\d+\.\d+)\]" 
| eval result = sqrt( pow((x2 - x1),2) + pow((y2 - y1),2) )

[ap666]

Thank you ! That query went through. However, it did not do anything new. Can I add anything to the query to show me the result of the calculation?

[aoverfield]

I'm not sure what you want to do with the data or what the rest of the data/search looks like. You can see these new fields by using the table command (add the below to the end of your search). You can also use other transforming commands if you want to create a visualization with the data. https://docs.splunk.com/Splexicon:Transformingcommand 

| table start, end, x1, x2, y1, y2, result