Splunk Search

Compare values in two different json objects

vadim_osipov
Engager

Hello, 

This question has probably been asked and answered, but, I just can't seem to find a best solution; 

I have a search that returns N of similar json objects of approx type: 

{

name: "name", 

id: "id", 

somelist: [

    {

     name: "foo" 

     value: "bar" 

    },

    {

     name: "foo"

     value:  "baz" 

     },

   ...

]

}

 

where I want to compare the "somelist" part of every object to another object. In the end write out diff between them to separate column. 

 

Thanks a lot, 

Vadim

Labels (3)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

What would your expected results look like?

0 Karma

vadim_osipov
Engager

That's actually a good question. I don't yet know. My first guess would be to mark it somehow in a separate column. Like, "No Match", "Change" or some sort of that.

0 Karma

yuanliu
SplunkTrust
SplunkTrust

ITWhisperer is correct.  If you are unclear what the end result look like, any given solution could either backfire or not scratch the itch, so to speak.  There is a second clarification that is needed: what operation do you have in mind to "compare the 'somelist' part of every object to another object."

This said, I often find myself in the same struggle to untangle my own "requirements".  So I'd like to throw out some assumptions to help clarify the question.  In essence, you need to specify what this "another object" looks like.  Is it another list of name-value pair?  Or is it a scalar value list?  In both cases, I am assuming that the "another object" is a fixed object, not another indefinite set like "similar json objects" that your search returns. (Like ITWhisperer notes, comparing two indefinite sets can deplete resources very quickly.)

In the simplest case, if "another object" is a list of name-value pairs like the following:

{"anotherobject" : [{"name" : "foo", "value" : "bar"}, {"name" : "notfoo", "value" : "bar"}, ... {"name" : "foo", "value" : "biz"}]}

A meaningful comparison could be: Is the intersect between somelist and anotherobject empty?  This would be straightforward:

 

| set intersect [
  search that returns similar json objects
  | rename somelist{}.* as somelist_*
  | eval somepair = mvzip(somelist_name, somelist_value, "-")
  | stats values(somepair) as pairs
] [
  search or expression that returns another object
    | rename anotherobject{}.* as another_*
    | eval anotherpair = mvzip(another_name, another_value, "-")
    | stats values(anotherpair) as pairs
]

 

The second case, comparing somelist{}.value set (with a given somelist{}.name) to a scalar value set, is more convoluted.  Maybe you want to know which somelist{}.name gives non-empty intersection with this scalar value set?  Again, I will illustrate the simplest case, when you only want to know if somelist{}.name == "foo" gives non-empty intersection:

 

| set intersect [
  search that returns similar json objects
  | rename somelist{}.* as somelist_*
  | stats values(somelist_value) as values by somelist_name
  | where somelist_name == "foo"
  | table values
] [
  search or expression that returns another object
    | stats values(anotherobject_value) as values
]

 

It is worth noting that "set" command is used only for illustration purposes.  Depending on what you want to do next, there are often  more efficient methods.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Perhaps you need to work out what it is you are trying to achieve first. Trying to compare each event to every other event or even worse part of every event with every other part of every other event is not only difficult, it will lead to an exponential expansion of resource usage and quickly breach limits.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...